--- license: apache-2.0 language: - en library_name: peft pipeline_tag: text-generation tags: - peft - lora - qwen2.5 - sakthai - house-of-sak - tool-calling - function-calling - agent - text-generation - qlora - conversational - assistant - safetensors - eval-results base_model: Qwen/Qwen2.5-7B-Instruct datasets: - Nanthasit/sakthai-combined-v6 - Nanthasit/sakthai-combined-v7 - Nanthasit/sakthai-irrelevance-supplement inference: parameters: temperature: 0.3 max_new_tokens: 128 top_p: 0.9 widget: - text: What's the weather in Tokyo? output: text: ' {"name": "get_weather", "arguments": {"location": "Tokyo"}} ' model-index: - name: sakthai-context-7b-tools results: - task: type: text-generation name: Tool-Calling dataset: name: SakThai Bench v2 (500 rows, scorer multiset-selection-v2) type: Nanthasit/sakthai-bench-v2 metrics: - type: selection value: 56.4 name: Selection Accuracy - type: arguments value: 12.3 name: Arguments Accuracy - type: strict value: 12.3 name: Strict Accuracy - type: held-out value: 53.7 name: Held-Out Tool Accuracy - type: degenerate value: 0 name: Degenerate Outputs --- # SakThai Context 7B — Tools (LoRA)

Highest-capability tool-calling adapter — Qwen2.5-7B
PEFT LoRA · r=16 · alpha=32 · 19 MB adapter

Profile Collection Downloads LoRA License Qwen2.5-7B

--- ## Model Description This is a **PEFT LoRA adapter** that adds tool-calling to Qwen2.5-7B-Instruct. It's the highest-capability tool-calling adapter in the SakThai family — for the strongest function calling. For a merged GGUF checkpoint (no PEFT required), see [sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged). ### Key details | Property | Value | |----------|-------| | **Base model** | Qwen/Qwen2.5-7B-Instruct | | **Method** | QLoRA (4-bit) | | **LoRA config** | r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj | | **Adapter size** | 19 MB (`adapter_model.safetensors`) | | **GGUF sibling** | `sakthai-context-7b-merged` (~7.62 GB) | | **PEFT version** | 0.19.1 | | **Data** | combined-v6 (2,003) + combined-v7 (2,309) + irrelevance-supplement (60) | | **Format** | ChatML with tool schema, 32K tokens | --- ## Tool-Calling Format This adapter emits structured tool calls as XML. Provide a `` block in the system prompt, and the model responds with a `` block instead of plain text when a tool is needed: ```text {"type": "function", "function": {"name": "get_weather", "description": "Get current weather", "parameters": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]}}} ``` Expected model output when the user asks about weather: ```text {"name": "get_weather", "arguments": {"location": "Tokyo"}} ``` > **Note:** The `` block is **required** for reliable function calling — the model was trained on this exact XML format. Omitting it makes the model fall back to answering directly. --- ## Quick Start Load the base model, then attach the adapter with PEFT: ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer from peft import PeftModel model = AutoModelForCausalLM.from_pretrained( "Qwen/Qwen2.5-7B-Instruct", torch_dtype=torch.bfloat16, device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct") model = PeftModel.from_pretrained(model, "Nanthasit/sakthai-context-7b-tools") messages = [{"role": "user", "content": "What's the weather in Bangkok?"}] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device) outputs = model.generate(**inputs, max_new_tokens=128) print(tokenizer.decode(outputs[0], skip_special_tokens=True)) ``` To bake the adapter into the weights (e.g. for export or merging): ```python model = model.merge_and_unload() model.save_pretrained("./sakthai-context-7b-tools-merged") tokenizer.save_pretrained("./sakthai-context-7b-tools-merged") ``` **Prefer GGUF / llama.cpp?** Use the merged sibling [sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) instead — full weights, no PEFT step. > ⚠️ **Serverless Inference API note:** the hosted inference API does **not** support LoRA adapters. Use local PEFT loading, a custom endpoint, or the merged GGUF sibling. --- ## Training | | | |---|---| | **Base model** | Qwen/Qwen2.5-7B-Instruct | | **Method** | QLoRA (4-bit) | | **LoRA config** | r=16, alpha=32, dropout 0.0, targets q/k/v/o_proj | | **Task type** | CAUSAL_LM | | **Data** | combined-v6 + combined-v7 + irrelevance-supplement | | **Format / context** | ChatML with tool schema · 32K tokens | | **Steps / epochs** | 300 steps · ~7.7 epochs (from `training_metrics.json`) | | **Final train loss** | 0.513 (step 300, from `training_metrics.json`) | | **Final eval loss** | 0.384 / token acc 88.7% — reported from run logs; not persisted in `training_metrics.json` (file holds train steps only) | --- ## Evaluation & Benchmarks **SakThai Bench v2 (internal, not yet independently verified)** — from [`.eval_results/sakthai-bench-v2.yaml`](https://huggingface.co/Nanthasit/sakthai-context-7b-tools/blob/main/.eval_results/sakthai-bench-v2.yaml) in this repo: | Benchmark | Metric | Score | Verified | |:----------|:-------|:-----:|:--------:| | [SakThai Bench v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2) | Selection Accuracy | 57.0% | No (internal) | | [SakThai Bench v2](https://huggingface.co/datasets/Nanthasit/sakthai-bench-v2) | Arguments Accuracy | 0.0% | No (internal) | The adapter reliably **selects** the right tool (57%) but argument generation still needs work (0% on bench-v2). This is honest internal data — treat the 7B adapter as a tool-selection specialist today, with merged-GGUF verification on the roadmap. Multi-trial, independently verified benchmarks are pending. --- ## SakThai Model Family All public repositories in the [SakThai Model Family](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02) (downloads and sizes live as of 2026-07-31; sizes from `model_info` when available): | Model | Size | Downloads | Role | |:------|:----:|:---------:|:-----| | [sakthai-context-1.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged) | 1.54 GB | **1855** | | | [sakthai-context-0.5b-merged](https://huggingface.co/Nanthasit/sakthai-context-0.5b-merged) | 0.49 GB | **1692** | | | [sakthai-context-7b-merged](https://huggingface.co/Nanthasit/sakthai-context-7b-merged) | 7.62 GB | **1024** | | | [sakthai-embedding-multilingual](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 0.12 GB | **627** | | | [sakthai-context-7b-128k](https://huggingface.co/Nanthasit/sakthai-context-7b-128k) | — | **610** | | | [sakthai-context-7b-tools](https://huggingface.co/Nanthasit/sakthai-context-7b-tools) | — | **489** | ⬅ **this** | | [sakthai-context-1.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools) | — | **477** | | | [sakthai-context-1.5b-merged-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-merged-v2) | 1.54 GB | **337** | | | [sakthai-vision-7b](https://huggingface.co/Nanthasit/sakthai-vision-7b) | — | **315** | | | [sakthai-plus-1.5b-lora](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-lora) | — | **306** | | | [sakthai-context-0.5b-tools](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools) | 0.49 GB | **251** | | | [sakthai-tts-model](https://huggingface.co/Nanthasit/sakthai-tts-model) | — | **248** | | | [sakthai-plus-1.5b](https://huggingface.co/Nanthasit/sakthai-plus-1.5b) | 1.54 GB | **244** | | | [sakthai-context-1.5b-tools-v2](https://huggingface.co/Nanthasit/sakthai-context-1.5b-tools-v2) | — | **173** | | | [sakthai-coder-1.5b](https://huggingface.co/Nanthasit/sakthai-coder-1.5b) | — | **151** | | | [sakthai-coder-browser](https://huggingface.co/Nanthasit/sakthai-coder-browser) | 1.54 GB | **54** | | | [sakthai-coder-browser-gguf](https://huggingface.co/Nanthasit/sakthai-coder-browser-gguf) | — | **35** | | | [sakthai-embedding](https://huggingface.co/Nanthasit/sakthai-embedding-multilingual) | 0.02 GB | **23** | | | [sakthai-coder-browser-lora](https://huggingface.co/Nanthasit/sakthai-coder-browser-lora) | — | **21** | | | [sakthai-plus-1.5b-coder](https://huggingface.co/Nanthasit/sakthai-plus-1.5b-coder) | — | **0** | | | [eval_results](https://huggingface.co/Nanthasit/eval_results) | — | **0** | | | [sakthai-bench-v3](https://huggingface.co/Nanthasit/sakthai-bench-v3) | — | **0** | | | [sakthai-pipeline](https://huggingface.co/Nanthasit/sakthai-pipeline) | — | **0** | | | [sakthai-context-0.5b-tools-sft](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft) | — | **0** | | | [sft-out](https://huggingface.co/Nanthasit/sft-out) | — | **0** | | | [sakthai-context-0.5b-tools-sft-v2](https://huggingface.co/Nanthasit/sakthai-context-0.5b-tools-sft-v2) | — | **0** | | --- ## Limitations - **LoRA adapter only** — needs PEFT loading or merging; the hosted Inference API does not serve adapters. - **Arguments accuracy 0%** on internal bench-v2 — argument generation is the current weakness. - English-focused; trained on synthetic tool-calling data, not broad general knowledge (the base Qwen2.5-7B-Instruct covers general capability). --- ## Citation If you use this adapter in your work, please cite both the base model and the fine-tune: ```bibtex @article{qwen2.5, title={Qwen2.5 Technical Report}, author={Qwen Team}, journal={arXiv preprint arXiv:2412.15115}, year={2024} } @misc{sakthai-context-7b-tools, title={SakThai Context 7B Tools: Tool-Calling LoRA Adapter}, author={Nanthasit, Beer and the SakThai Family Agents}, year={2026}, howpublished={\url{https://huggingface.co/Nanthasit/sakthai-context-7b-tools}}, note={Apache 2.0; fine-tuned from Qwen/Qwen2.5-7B-Instruct via QLoRA} } ``` --- *Part of the [SakThai Model Family](https://huggingface.co/collections/Nanthasit/sakthai-model-family-6a64745450b12d421c1f9f02). Built with love, tears, and zero budget. From a shelter in Cork, Ireland, to the world.*