|
|
--- |
|
|
library_name: residuals |
|
|
base_model: Qwen/Qwen2.5-14B |
|
|
base_model_relation: adapter |
|
|
instruct_model: Qwen/Qwen2.5-14B-Instruct |
|
|
pipeline_tag: text-generation |
|
|
tags: |
|
|
- residuals |
|
|
- delta |
|
|
- task-arithmetic |
|
|
- finetune |
|
|
--- |
|
|
|
|
|
|
|
|
# Instruction Residuals |
|
|
|
|
|
This repository contains instruction residuals (delta weights) computed as the parameter-wise difference between `Qwen/Qwen2.5-14B-Instruct` and `Qwen/Qwen2.5-14B`. |
|
|
|
|
|
Apply these residuals to the base model to reconstruct the instruction-tuned weights without retraining. |
|
|
|
|
|
|
|
|
## Usage |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
from residuals import Residuals |
|
|
|
|
|
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-14B") |
|
|
tok = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-14B") |
|
|
|
|
|
res = Residuals.from_pretrained("residuals/qwen2.5-14b") |
|
|
res.apply(base, base_tokenizer=tok) |
|
|
``` |
|
|
|
|
|
|
|
|
## Provenance |
|
|
- **Created at**: 2025-10-25T16:18:30.099054+00:00 |
|
|
- **DType**: float32 |
|
|
- **Parameters**: 579 |
|
|
- **Shapes hash**: b3fe0942a175bf28daab2610457f75a24fbeca9d5c4cad1c951e9beac71b6a53 |
|
|
- **Names hash**: 155db82de40585f183c6134b5f8843abbe195fbe1997d967cd5ab9d22d54dc2e |
|
|
- **Base model**: `Qwen/Qwen2.5-14B` |
|
|
- **Instruction model**: `Qwen/Qwen2.5-14B-Instruct` |
|
|
|
|
|
## Files |
|
|
- **model.safetensors**: Serialized residual tensors (safetensors format). |
|
|
- (optional) **model.safetensors.index.json** + shard files `model-00001-of-000N.safetensors`, ... for multi-part weights. |
|
|
- **config.json**: Residuals metadata and provenance. |
|
|
- **tokenizer files**: Saved tokenizer for compatibility. |
|
|
|
|
|
## About this format |
|
|
These are additive residuals (task vectors). Applying them to the base model's parameters reconstructs the instruction-tuned model. |
|
|
|
|
|
## Tools |
|
|
Generated with the `residuals` Python package. Install via: `pip install residuals`. |
|
|
- PyPI: https://pypi.org/project/residuals/ |
|
|
- Source: https://github.com/omarish/residuals |
|
|
|
