README.md

metadata

base_model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - finance
  - banking
  - rag
  - conversational-ai
  - lora
license: apache-2.0
language:
  - en
library_name: transformers
pipeline_tag: text-generation

Banking AI Assistant - Llama 3.2 1B Fine-tuned

A specialized banking and financial AI assistant fine-tuned on the T2-RAGBench dataset for conversational RAG tasks. This model excels at analyzing financial documents, answering banking-related questions, and providing detailed insights from financial reports.

Model Details

• Developed by: Akhenaton
• Model Type: Causal Language Model (Llama 3.2 1B)
• License: Apache 2.0
• Base Model: unsloth/llama-3.2-1b-instruct-unsloth-bnb-4bit
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Framework: Unsloth + Hugging Face TRL
• Quantization: 4-bit (BitsAndBytes)

Training Details

Dataset

• Source: G4KMU/t2-ragbench (ConvFinQA subset)
• Size: 32,908 context-independent QA pairs from 9,000+ financial documents
• Domains: FinQA, ConvFinQA, VQAonBD, TAT-DQA
• Focus: Financial documents with text and tables from SEC filings

Training Configuration

LoRA Parameters:
  r: 16
  lora_alpha: 16
  lora_dropout: 0
  target_modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]

Training Setup:
  max_seq_length: 2048
  per_device_train_batch_size: 2
  gradient_accumulation_steps: 4
  max_steps: 60
  learning_rate: 2e-4
  optimizer: adamw_8bit
  lr_scheduler_type: cosine
  weight_decay: 0.01

Intended Use

Primary Use Cases

• Financial Document Analysis: Extract insights from financial reports, SEC filings, and earnings statements
• Banking Q&A: Answer questions about financial concepts, regulations, and banking operations
• Conversational RAG: Provide context-aware responses based on financial document context
• Financial Research: Assist with financial research and analysis tasks

Conversation Format

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a specialized banking AI assistant. Analyze financial documents and provide accurate, detailed answers based on the given context. Focus on numerical accuracy and financial terminology.<|eot_id|><|start_header_id|>user<|end_header_id|>

Financial Document Context:
{context}

Question: {question}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{response}<|eot_id|>

Usage

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("Akhenaton/sft_banking_model")
tokenizer = AutoTokenizer.from_pretrained("Akhenaton/sft_banking_model")

# Prepare conversation
messages = [
    {"role": "user", "content": "Explain the key financial metrics in quarterly earnings."}
]

# Generate response
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=128, temperature=1.5, min_p=0.1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

With Unsloth (Recommended - 2x faster)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    "Akhenaton/sft_banking_model",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True
)
FastLanguageModel.for_inference(model)  # Enable fast inference

Available Formats

This model is available in multiple quantization formats:

• q4_k_m: Recommended for most use cases
• q8_0: Higher quality, more resource intensive
• q5_k_m: Balanced quality and efficiency
• f16: Full precision for maximum accuracy

Performance

• Training Speed: 2x faster with Unsloth optimization
• Memory Efficiency: 4-bit quantization reduces VRAM requirements
• Inference Speed: Optimized for fast response generation
• Accuracy: Specialized for financial domain with >80% context-independent Q&A capability

Limitations

• Domain Specific: Optimized for financial/banking content, may have reduced performance on general topics
• Training Size: Limited to 60 training steps - further training may improve performance
• Context Length: Maximum sequence length of 2048 tokens
• Language: English only
• Numerical Reasoning: While improved for financial calculations, complex mathematical operations may require verification

Ethical Considerations

• Financial Advice: This model should not be used as a substitute for professional financial advice
• Data Source: Trained on public SEC filings and financial documents
• Bias: May reflect biases present in financial reporting and documentation
• Verification: Always verify numerical calculations and financial information from authoritative sources

Citation

If you use this model in your research or applications, please consider citing:

@misc{akhenaton2025sft_banking_model,
  author = {Akhenaton},
  title = {Banking AI Assistant - Llama 3.2 1B Fine-tuned},
  year = {2025},
  url = {https://huggingface.co/Akhenaton/sft_banking_model},
  note = {Fine-tuned with Unsloth on T2-RAGBench dataset}
}

Acknowledgments

• Unsloth Team for the optimized training framework
• Meta AI for the Llama 3.2 base model
• G4KMU for the T2-RAGBench dataset
• Hugging Face for the transformers library and model hosting

---

This model was trained 2x faster with Unsloth and Hugging Face's TRL library.