aamanlamba/phi3-payments-finetune

Phi-3 Mini Fine-tuned for Payments Domain

This is a fine-tuned version of Microsoft's Phi-3-Mini-4k-Instruct model, adapted for generating natural language descriptions of payment transactions using LoRA (Low-Rank Adaptation).

Model Description

This model converts structured payment transaction data into clear, customer-friendly language. It was fine-tuned using LoRA on a synthetic payments dataset covering various transaction types.

Training Data

The model was trained on a dataset of 500+ synthetic payment transactions including:

• Standard payments (ACH, wire transfer, credit/debit card)
• Refunds (full and partial)
• Chargebacks
• Failed/declined transactions
• International transfers with currency conversion
• Transaction fees
• Recurring payments/subscriptions

Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = "microsoft/Phi-3-mini-4k-instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype="auto",
    device_map="auto"
)

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "aamanlamba/phi3-payments-finetune")
tokenizer = AutoTokenizer.from_pretrained(base_model)

# Generate description
prompt = """<|system|>
You are a financial services assistant that explains payment transactions in clear, customer-friendly language.<|end|>
<|user|>
Convert the following structured payment information into a natural explanation:

inform(transaction_type[payment], amount[1500.00], currency[USD], sender[Acme Corp], receiver[Global Supplies Inc], status[completed], method[ACH], date[2024-10-27])<|end|>
<|assistant|>
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.7)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Expected output:

Your ACH payment of $1,500.00 to Global Supplies Inc was successfully completed on October 27, 2024.

Training Details

Training Configuration

• Base Model: microsoft/Phi-3-mini-4k-instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• LoRA Rank: 16
• LoRA Alpha: 32
• Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Quantization: 8-bit (training), float16 (inference)
• Training Epochs: 3
• Learning Rate: 2e-4
• Batch Size: 1 (with 8 gradient accumulation steps)
• Hardware: NVIDIA RTX 3060 (12GB VRAM)
• Training Time: ~35-45 minutes

Training Loss

• Initial Loss: ~3.5-4.0
• Final Loss: ~0.9-1.2
• Validation Loss: ~1.0-1.3

Model Size

• LoRA Adapter Size: ~15MB (only the adapter weights, not the full model)
• Full Model Size: ~7GB (when combined with base model)

Supported Transaction Types

1. Payments: Standard payment transactions
2. Refunds: Full and partial refunds
3. Chargebacks: Dispute and chargeback processing
4. Failed Payments: Declined or failed transactions with reasons
5. International Transfers: Cross-border payments with currency conversion
6. Fees: Transaction and processing fees
7. Recurring Payments: Subscriptions and scheduled payments
8. Reversals: Payment reversals and adjustments

Limitations

• Trained on synthetic data - may require additional fine-tuning for production use
• Optimized for English language only
• Best performance on transaction patterns similar to training data
• Not suitable for handling real financial transactions without human oversight
• Should not be used as the sole system for financial communication

Ethical Considerations

• This model was trained on synthetic, anonymized data only
• Does not contain any real customer PII or transaction data
• Should be validated for accuracy before production deployment
• Implement human review for customer-facing financial communications
• Consider regulatory compliance (PCI-DSS, GDPR, etc.) in your jurisdiction

Intended Use

Primary Use Cases:

• Generating transaction descriptions for internal systems
• Creating customer-friendly payment notifications
• Automating payment communication drafts (with human review)
• Training and demonstration purposes
• Research in financial NLP

Out of Scope:

• Direct customer communication without review
• Real-time transaction processing without validation
• Compliance-critical communications
• Medical or legal payment descriptions

How to Cite

If you use this model in your research or application, please cite:

@misc{phi3-payments-finetuned,
  author = {aamanlamba},
  title = {Phi-3 Mini Fine-tuned for Payments Domain},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/aamanlamba/phi3-payments-finetune}}
}

Training Code

The complete training code and dataset generation scripts are available on GitHub:

• Repository: github.com/aamanlamba/phi3-tune-payments
• Includes: Dataset generator, training scripts, testing utilities, and deployment guides

Acknowledgements

• Base model: Microsoft Phi-3-Mini-4k-Instruct
• Fine-tuning method: LoRA: Low-Rank Adaptation of Large Language Models
• Training framework: HuggingFace Transformers + PEFT
• Inspired by: NVIDIA AI Workbench Phi-3 Fine-tuning Example

License

This model is released under the MIT license, compatible with the base Phi-3 model license.

Contact

For questions or issues, please open an issue on the model repository or contact the author.

---

Note: This is a demonstration model. Always validate outputs for accuracy before use in production financial systems.