Instructions to use durganani60/qwen2.5-0.5b-financial-adapted with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use durganani60/qwen2.5-0.5b-financial-adapted with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="durganani60/qwen2.5-0.5b-financial-adapted")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("durganani60/qwen2.5-0.5b-financial-adapted")
model = AutoModelForCausalLM.from_pretrained("durganani60/qwen2.5-0.5b-financial-adapted", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use durganani60/qwen2.5-0.5b-financial-adapted with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use durganani60/qwen2.5-0.5b-financial-adapted with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "durganani60/qwen2.5-0.5b-financial-adapted"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "durganani60/qwen2.5-0.5b-financial-adapted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/durganani60/qwen2.5-0.5b-financial-adapted

SGLang

How to use durganani60/qwen2.5-0.5b-financial-adapted with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "durganani60/qwen2.5-0.5b-financial-adapted" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "durganani60/qwen2.5-0.5b-financial-adapted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "durganani60/qwen2.5-0.5b-financial-adapted" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "durganani60/qwen2.5-0.5b-financial-adapted",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use durganani60/qwen2.5-0.5b-financial-adapted with Docker Model Runner:
```
docker model run hf.co/durganani60/qwen2.5-0.5b-financial-adapted
```

Qwen2.5-0.5B Financial Domain Adapted LLM

This model is a domain-adapted version of Qwen2.5-0.5B specifically tailored for financial NLP tasks. It features a custom Byte-Level BPE Tokenizer integrated directly into the base architecture via Tokenizer Surgery and aligned using TRL SFTTrainer with PEFT/LoRA on Apple Silicon (MPS).

Model Details

Model Description

Standard LLM tokenizers often fragment specialized financial terminology (such as ticker symbols like $NVDA, key metrics like EBITDA/CAGR, and currency pairs like EUR/USD) into excessive subwords, leading to increased prompt inflation and potential [UNK] errors.

This model addresses these issues by:

Extending the base vocabulary with custom financial domain tokens.
Resizing the underlying embedding matrix ($E \in \mathbb{R}^{V_{new} \times d}$) and language model head.
Conducting continued pre-training using LoRA to train the new token embeddings while preserving base model capabilities.

Developed by: Durga
Model type: Causal Language Model (Transformer Decoder)
Language(s) (NLP): English (Financial Domain)
License: Apache 2.0
Finetuned from model: Qwen/Qwen2.5-0.5B

Uses

Direct Use

Financial text generation and completion.
Processing financial filings (SEC 10-K, 10-Q), earnings transcripts, and analyst reports.
Reduced context fragmentation for domain-specific prompt engineering.

Out-of-Scope Use

Generating binding financial advice, automated trading decisions, or official financial audits without human oversight.

Bias, Risks, and Limitations

Domain Specificity: The model has been continued pre-trained on a focused financial corpus. It is designed for domain text compression and processing rather than general open-domain chat.
Accuracy: Outputs should be independently verified when used for quantitative or regulatory compliance tasks.

How to Get Started with the Model

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load model and custom adapted tokenizer
model_id = "durganani60/qwen2.5-0.5b-financial-adapted"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# Test prompt containing financial domain tokens
prompt = "Q3 2024 financial report: $NVDA recorded EBITDA growth above 14.5% with expanding FCF margin."

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50, do_sample=True, temperature=0.7)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
'''

Downloads last month: 89

Safetensors

Model size

0.6B params

Tensor type

F16

Model tree for durganani60/qwen2.5-0.5b-financial-adapted

Base model

Qwen/Qwen2.5-0.5B

Finetuned

(742)

this model