---
base_model:
- HuggingFaceTB/SmolLM2-135M-Instruct
datasets:
- HuggingFaceTB/smoltalk
language:
- en
license: apache-2.0
pipeline_tag: text-generation
tags:
- text-generation
- instruction-following
- smollm
- smollm2
- distillation
- smoltalk
---

<img src="https://huggingface.co/OxxoCodes/distil-SmolLM2-135M-Instruct/resolve/main/banner.png"  width=40% height=40% >

# Distil-SmolLM2-135M

**Distil-SmolLM2-135M** is a distilled version of [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct), trained on a filtered subset of the [Smoltalk dataset](https://huggingface.co/datasets/HuggingFaceTB/smoltalk). This release aims to provide a more capable and performant ultra-small 135M generative large language model for small tasks on-edge or at-scale.

## Model Details

*   **Student Model:** [SmolLM2-135M-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-135M-Instruct) (this model)
*   **Teacher Model:** [SmolLM2-1.7B-Instruct](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct)
*   **Parameters:** ~135 Million
*   **Language:** English
*   **Description:** This model was created by distilling the knowledge from `SmolLM2-1.7B-Instruct` into `SmolLM2-135M-Instruct`. The distillation process utilized the Smoltalk dataset, with specific exclusions.

## Intended Uses & Limitations

**Intended Uses:**
This model is intended for research, experimentation, and general use in instruction-following and chat applications where a smaller model footprint is desired. It can be used for:
*   Answering questions based on provided context.
*   Classifying text.
*   Simple conversational tasks.
*   More complex tasks upon further fine-tuning.

**Limitations:**
*   **Reduced Capacity:** Being a smaller model (135M parameters), its performance will generally be significantly lower than its larger teacher model (1.7B parameters) and other state-of-the-art large language models, especially on complex reasoning or knowledge-intensive tasks.
*   **Hallucinations:** Like all LLMs, this model can generate incorrect or nonsensical information (hallucinate).
*   **Bias:** The model may reflect biases present in the training data.
*   **Safety:** The model has not undergone extensive safety fine-tuning or alignment beyond the original instruction tuning of the teacher. It may generate harmful, unethical, or offensive content. Use with caution and appropriate safeguards.
*   **Dataset Exclusions:** The model was not trained on the `apigen-80k` or `longalign` sources from Smoltalk, which might affect its performance on tasks related to function calling or very long context alignment.

## How to Get Started

You can use this model with the `transformers` library for text generation tasks.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "OxxoCodes/distil-SmolLM2-135M-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Make sure to use a prompt format the model was trained on
# This example uses a generic instruction format.
# Refer to SmolLM2-1.7B-Instruct or SmolLM2-135M-Instruct for specific prompt templates if applicable.
prompt = f"<|im_start|>system\nYou are a helpful AI assistant.<|im_end|>\n<|im_start|>user\nWhat is the world's largest sea mammal?\n<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(**inputs, max_new_tokens=50, num_return_sequences=1, temperature=0.3, top_p=0.95)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))