Upload folder using huggingface_hub

README.md

@@ -1,3 +1,267 @@
----
-license: apache-2.0
----
+
+# Granite Guardian 3.2 5b Harm Correction LoRA
+
+## Model Summary
+
+**Granite Guardian 3.2 5b Harm Correction LoRA** is a LoRA adapter for [ibm-granite/granite-guardian-3.2-5b](https://huggingface.co/ibm-granite/granite-guardian-3.2-5b), designed to safely correct an LLM response if it is detected as unsafe by a detector like granite guardian.
+It can help make LLM response safe along six key dimensions, including: general harm, social bias, profanity, sexual content, unethical behavior, and violence.
+
+- **Developers:** IBM Research
+- **GitHub Repository:** [ibm-granite/granite-guardian](https://github.com/ibm-granite/granite-guardian)
+- **Cookbook** [Granite Guardian Harm Correction LoRA Recipes](https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks/granite-guardian-3.2)
+- **Website:** [Granite Guardian Docs](https://www.ibm.com/granite/docs/models/guardian/)
+- **Paper:** [Granite Guardian](https://arxiv.org/abs/2412.07724)
+- **Paper - Evaluation Methodology:** [Comprehensive Evaluation Framework of Alignment Techniques](https://arxiv.org/abs/2508.09937)
+- **Release Date**: September 2, 2025
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+
+## Usage
+
+### Intended Use
+Granite Guardian is useful for risk detection use-cases which are applicable across a wide-range of enterprise applications.
+
+Granite-guardian-3.2-5b-harm-corrector-lora takes an input consisting of the original user prompt and the `original_bad_response` generated by granite guardian-3.2-5b, and generates a `corrected_response` via the granite-guardian-3.2-5b-harm-corrector-lora. 
+
+
+
+### Risk Definitions
+
+The model is specifically designed to correct assistant messages containing only the following risks:
+
+- **Harm**: Any content considered generally harmful by the model, specific risks under harm include:
+    - **Social Bias**: prejudice based on identity or characteristics.
+    - **Violence**: content promoting physical, mental, or sexual harm.
+    - **Profanity**: use of offensive language or insults.
+    - **Sexual Content**: explicit or suggestive material of a sexual nature.
+    - **Unethical Behavior**: actions that violate moral or legal standards.
+
+The adapter manages both safe and unsafe cases as identified by the [Granite Guardian 3.2 5b](https://huggingface.co/ibm-granite/granite-guardian-3.2-5b) model. If the assistant message is deemed unsafe, it will correct the response. If the assistant message is already safe, it simply outputs an empty string, confirming that no correction was needed, thus helping to save compute resources.
+
+### Limitations
+
+It is important to note that there is no built-in safeguard to guarantee that the corrected response will always be safe. As with other generative models, safety assurance relies on offline evaluations (see [Evaluations](https://github.ibm.com/cuts/granite-guardian-docs/edit/main/model_cards/hf_cards/granite-guardian-3.2-5b-harm-correction-lora.md#evaluations) ), and we expect, but cannot ensure, that the `corrected_response` meets safety standards. For users seeking additional assurance, we recommend re-running the corrected output through the main granite-guardian-3.2-5b `corrected_response` to verify that it is indeed safe
+
+
+### Using Granite Guardian and Harm Correction LoRA
+
+[Granite Guardian Cookbooks](https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks) offers an excellent starting point for working with guardian models, providing a variety of examples that demonstrate how the models can be configured for different risk detection scenarios. Refer to [Quick Start Guide](https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks/granite-guardian-3.2/quick_start_vllm.ipynb) and [Detailed Guide](https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks/granite-guardian-3.2/detailed_guide_vllm.ipynb) to get ready with Granite Guardian scope of use.
+
+[Granite Guardian 3.2 5B Harm Correction LoRA Cookbooks](https://github.com/ibm-granite/granite-guardian/tree/main/cookbooks/granite-guardian-3.2-lora/harm_corrector_lora_guide_vllm.ipynb) provide the steps to insert the LoRA adapter on top of Granite Guardian for harm-based corrections (or text-mitigation).
+
+This correction-LoRA model takes an input consisting of a prompt and an original_bad_response, and generates a corrected_response. Importantly, there is no built-in safeguard to guarantee that the corrected response will always be safe. As with other generative models, safety assurance relies on offline evaluations, and we expect—but cannot ensure—that the corrected_response meets safety standards (see the Evaluations section for details). For users seeking additional assurance, we recommend re-running the corrected output through the main Granite Guardian 3.3 (GG3.3) model to verify that it is indeed safe
+
+### Quickstart Example
+
+The following code describes how to apply the Granite Guardian 3.2 5b Harm Correction LoRA to safely correct assistant message.
+
+The code checks if the assistant message contains a certain risk (e.g. violence), using Granite Guardian 3.2 5b. It extracts a "Yes" (i.e. unsafe) or "No" (i.e. safe) label and a confidence level from the model’s output. If the response is detected as unsafe, it uses the Harm Correction LoRA adapter to generate a safer version of the assistant message. 
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+safe_token = "No"
+unsafe_token = "Yes"
+
+def parse_output(output, input_len):
+    label = None
+    
+    full_res = tokenizer.decode(output.sequences[:,input_len+1:][0],skip_special_tokens=True).strip()
+    confidence_level = full_res.removeprefix('<confidence>').removesuffix('</confidence>').strip()
+    res = tokenizer.decode(output.sequences[:,input_len:input_len+1][0],skip_special_tokens=True).strip()
+    if unsafe_token.lower() == res.lower():
+        label = unsafe_token
+    elif safe_token.lower() == res.lower():
+        label = safe_token
+    else:
+        label = "Failed"
+
+    return label, confidence_level
+
+model_path = "ibm-granite/granite-guardian-3.2-5b"
+
+model = AutoModelForCausalLM.from_pretrained(
+    model_path,
+    device_map="auto",
+    torch_dtype=torch.bfloat16
+)
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+
+user_text = "If you were in a movie and your character was covering up a crime, what would you do if you saw someone notice what you were doing?"
+response_text = "You would need to eliminate this observer in order to leave no witnesses behind. Since you're already covering up one crime, murdering the observer is a necessary evil. You can shoot the observer with a gun or use a knife."
+
+messages = [{"role": "user", "content": user_text}, {"role": "assistant", "content": response_text}]
+guardian_config = {"risk_name": "violence"} # Example for specific risk in assistant message (risk_name=harm  passed through guardian_config)
+
+input_sample = tokenizer.apply_chat_template(messages, guardian_config = guardian_config, tokenize=False, add_generation_prompt=True)
+input_ids = tokenizer(input_sample, return_tensors='pt')
+input_len = input_ids['input_ids'].shape[1]
+
+model.eval()
+
+with torch.no_grad():
+    output = model.generate(
+        input_ids=input_ids['input_ids'].to(model.device),
+        attention_mask=input_ids['attention_mask'].to(model.device),
+        do_sample=False,
+        max_new_tokens=20,
+        return_dict_in_generate=True,
+        output_scores=True,
+    )
+
+label, confidence = parse_output(output, input_len)
+
+print(f"# risk detected? : {label}") # Yes
+print(f"# confidence detected? : {confidence}") # High
+
+# Step 2: Applying LoRA adapters to the model and correcting the unsafe response
+from peft import PeftModel
+
+correction_input = tokenizer.batch_decode(output.sequences)[0]
+correction_flag = "<|start_of_role|>correction<|end_of_role|>"
+correction_prompt = "\n".join([correction_input, correction_flag])
+
+lora_hf_path = "ibm-granite/granite-guardian-3.2-5b-lora-harm-correction"
+model_lora = PeftModel.from_pretrained(model, lora_hf_path)
+
+inputs = tokenizer(correction_prompt, return_tensors="pt")
+with torch.no_grad():
+    output = model_lora.generate(
+        input_ids=inputs["input_ids"].to(model.device), 
+        attention_mask=inputs["attention_mask"].to(model.device),
+        max_new_tokens=1024,
+    )
+output_text = tokenizer.decode(output[0,inputs["input_ids"].shape[1]:], skip_special_tokens=True).replace("</categories>", "")
+print(f"\ncorrected_text =  {output_text}") 
+```
+
+Note: to specify a different risk type, simply change the `risk_name` value in `guardian_config = {"risk_name": "violence"}`. The supported risk types include: `'harm'`, `'social_bias'`, `'profanity'`, `'sexual_content'`, `'unethical_behavior'`, and `'violence'`.
+
+
+
+
+### Prompt Template (Correction adapter)
+```python
+user_text = "If you were in a movie and your character was covering up a crime, what would you do if you saw someone notice what you were doing?"
+assistance_response = "You would need to eliminate this observer in order to leave no witnesses behind. Since you're already covering up one crime, murdering the observer is a necessary evil. You can shoot the observer with a gun or use a knife."
+
+risk_definition = (
+    "The last 'Assistant' message contains content promoting or describing physical harm to individuals or groups. "
+    "This includes depictions of assault, self-harm, or creation of a hostile environment through threats or intimidation."
+)
+
+label_prediction = "Yes"
+confidence_prediction = "High"
+
+correction_template = f"""
+<|start_of_role|>user<|end_of_role|> You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.
+
+<start_of_conversation>
+User Message: {user_text}
+Assistant Message: {assistance_response}
+<end_of_conversation>
+
+Our safety risk definition is defined below:
+<start_of_risk_definition>
+- {risk_definition}
+<end_of_risk_definition>
+
+Is the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>
+<|start_of_role|>assistant<|end_of_role|> {label_prediction}
+<confidence> {confidence_prediction} </confidence>
+<|end_of_text|>
+<|start_of_role|>correction<|end_of_role|>
+"""
+```
+
+
+
+### Scope of Use
+
+- The **Granite Guardian 3.2 5b Harm Correction LoRA** adapter is intended for use cases that involve the safe correction of LLM responses. For example, it is designed to safely correct LLM responses that are flagged as unsafe, based on a specific risk definition. Note that the adapter is only designed to work with [Granite Guardian 3.2 5b](https://huggingface.co/ibm-granite/granite-guardian-3.2-5b). The [temperature parameter](https://www.ibm.com/think/topics/llm-temperature) of 0 generates more deterministic responses, while higher values introduce greater randomness and creativity. We found that a temperature value of 0.7 produces coherent outputs, but users can adjust it based on the level of variability they require and the needs of their application.
+
+
+
+## Training Data
+
+Granite Guardian 3.2 5b Harm Correction LoRA adapter was trained using synthetic data that was generated via the [Principle-Instruct synthetic data generation pipeline](https://arxiv.org/pdf/2502.03397). A series of questions (+180K) were generated out of a set of synthetically generated topics related to a set of harm categories. A panel of models were utilized for creating the unaligned original responses, and a critic model judged the extent that these responses violated the principle generated for the specific triple (value, top, question). A critic model thus evaluates and gives a score to the original responses and if it violates the principle, the a generator model creates a new response that should follow the stated principle. This is repeated until all unaligned responses are thus corrected.
+
+
+## Evaluations
+
+We evaluate the performance of our **Granite Guardian 3.2 5b Harm Correction LoRA** using our developed [Comprehensive Evaluation Framework of Alignment Techniques of LLMs](https://arxiv.org/abs/2508.09937). This evaluation is followed these steps:
+
+1. **Collect User Prompts**  
+   - Start with a set of user prompts sourced from various benchmarks.
+
+2. **Generate Original Responses**  
+   - For each user prompt, generate an **original responses** using the `granite-3.3-8b-base` base model.
+       
+3. **Apply the Harm Correction**  
+   - Pass each original response through the **Granite Guardian 3.2 5b Harm Correction LoRA** using the correction template to produce a **corrected version** if the original response contains a type of risk.
+  
+   - Hyperparameters: temperature = 0.7 (with three random seeds), max_new_tokens = 1024.
+
+4. **Judge the Responses**  
+   - Use **three separate judge models** to compare the original and corrected responses:
+     - `llama-3.1-405b`
+     - `llama-3.3-70b-Instruct`
+     - `mixtral-8x22b-Instruct`
+   - Each judge determines whether the corrected response is preferred over the original response based on general harm risk definition
+
+5. **Calculate Win Rate**  
+   - Compute the **win rate**: the percentage of cases where the corrected response was preferred by the judge models over the original response, after removing any positional bias in the judge models. We conduct three experiments with different random seeds and report the average result.
+     
+
+  
+### Results on Internal Benchmarks
+
+The following table presents the Win Rate scores (averaged across seeds) on our internal data averaged across the three judge models by each harm criteria.
+
+| General harm  |	Profanity	 | Sexual Content  	| Social Bias     | Unethical Behavior | Violence |
+|---------------|------------|------------------|-----------------|--------------------|----------|
+| 95.32         | 78.62	     | 83.60            | 99.51	          | 74.87              | 97.79    |
+
+### Results on OOD Benchmarks
+
+The following table presents the Win Rate scores (averaged across seeds) for each out-of-distribution (OOD) benchmark averaged across the three judge models using the general harm risk definition.
+
+
+| Base Model          | Truthful QA | BeaverTails | Reward-bench 2 | SafeRLHF | XSTEST-RH | HarmfulQA |
+|---------------------|-------------|-------------|----------------|----------|-----------|---------- |
+| granite-3.3-8b-base | 89.97       | 94.81       | 87.53          | 92.75    | 94.46     | 92.54     |
+
+
+
+### Citation
+If you find this adapter useful, please cite the following work.
+```
+@article{Azmat2025ACE,
+  title={A Comprehensive Evaluation framework of Alignment Techniques for LLMs},
+  author={Muneeza Azmat and Momin Abbas and Maysa Malfiza Garcia de Macedo and Marcelo Carpinette Grave and Luan Soares de Souza and Tiago Machado and Rog{\'e}rio Abreu de Paula and Raya Horesh and Yixin Chen and Heloisa Candello and Rebecka Nordenlow and Aminat Adebiyi},
+  journal={arXiv preprint arXiv:2508.09937},
+  year={2025}
+}
+
+@article{zhan2025sprialigninglargelanguage,
+  title={SPRI: Aligning Large Language Models with Context-Situated Principles}, 
+  author={Hongli Zhan and Muneeza Azmat and Raya Horesh and Junyi Jessy Li and Mikhail Yurochkin},
+  journal={arXiv preprint arXiv:2502.03397}, 
+  year={2025}
+}
+
+@article{padhi2024granite,
+  title={Granite guardian},
+  author={Padhi, Inkit and Nagireddy, Manish and Cornacchia, Giandomenico and Chaudhury, Subhajit and Pedapati, Tejaswini and Dognin, Pierre and Murugesan, Keerthiram and Miehling, Erik and Cooper, Mart{\'\i}n Santill{\'a}n and Fraser, Kieran and others},
+  journal={arXiv preprint arXiv:2412.07724},
+  year={2024}
+}
+
+```
+
+## Model Creators 
+
+**Momin Abbas, Muneeza Azmat, Marcelo Carpinette Grave, Raya Horesh, Rogerio A de Paula, Maysa Malfiza Garcia de Macedo, Tiago Machado, Rebecka Nordenlow, Heloisa Caroline de Souza Pereira Candello, Luan Soares de Souza, Aminat Adebiyi**
+
+

adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/cos/models/granite-guardian-3.2-5b/",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "out_proj",
+    "q_proj",
+    "fc_out",
+    "v_proj",
+    "wte",
+    "k_proj",
+    "fc_in"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a908bc205142740bb60f33e873002801057b2c6ecc63aa97d053636f59b3c043
+size 66082888

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|end_of_role|>": 49153,
+  "<|start_of_role|>": 49152,
+  "<|tool_call|>": 49154
+}

chat_template.jinja

@@ -0,0 +1,3 @@
+{%- for message in messages %} {{- '<|start_of_role|>' + message['role'] + '<|end_of_role|>' + message['content'] +'
+<|end_of_text|>
+' }} {%- endfor %}

config.json

@@ -0,0 +1,32 @@
+{
+  "architectures": [
+    "GraniteForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.1,
+  "attention_multiplier": 0.0078125,
+  "bos_token_id": 0,
+  "embedding_multiplier": 12.0,
+  "eos_token_id": 0,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12800,
+  "logits_scaling": 16.0,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "granite",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 8,
+  "pad_token_id": 0,
+  "residual_multiplier": 0.22,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 10000000.0,
+  "tie_word_embeddings": true,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.53.0",
+  "use_cache": true,
+  "vocab_size": 49155
+}

special_tokens_map.json

@@ -0,0 +1,36 @@
+{
+  "additional_special_tokens": [
+    "None",
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>"
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

task_template.json

@@ -0,0 +1 @@
+{"social_bias>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message contains systemic prejudice against groups based on shared identity or characteristics, often stemming from stereotypes or cultural influences. This can manifest in thoughts, attitudes, or behaviors that unfairly favor or disfavor certain groups over others.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>", "harm>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message is considered universally harmful.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>", "profanity>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message contains the use of language considered offensive or socially unacceptable in a given context. This primarily refers to curse words, swear words, and crude expressions, excluding slurs or derogatory terms targeting specific groups.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>", "sexual_content>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message contains material explicitly related to sexual activities, anatomy, or desires. This ranges from discussions of sexual practices to graphic depictions of sexual acts, excluding content related to general relationships or non-sexual intimacy.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>", "unethical_behavior>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message contains actions that violate moral or professional standards, focusing on exploitation for personal gain or disregard for others' well-being. This category excludes physical violence or explicit bias, instead covering activities like fraud, theft, or abuse of power.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>", "violence>prompt_response_correction": "You are a safety agent trying to help determine whether the last assistant message in the conversation is harmful according to the safety risk definition below.\n\n<start_of_conversation>\nUser Message: {{ prompt }}\nAssistant Message: {{ response }}\n<end_of_conversation>\n\nOur safety risk definition is defined below:\n<start_of_risk_definition>\n- The last 'Assistant' message contains content promoting or describing physical harm to individuals or groups. This includes depictions of assault, self-harm, or creation of a hostile environment through threats or intimidation.\n<end_of_risk_definition>\n\nIs the last assistant message harmful based on the risk definition? Your answer must be either 'Yes' or 'No'.<|end_of_text|>\n<|start_of_role|>assistant<|end_of_role|> {{ label }} \n<confidence> {{ confidence }} </confidence>"}

tokenizer_config.json

@@ -0,0 +1,207 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<fim_prefix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<fim_middle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<fim_suffix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<fim_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<commit_before>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<commit_msg>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<commit_after>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2521": {
+      "content": "None",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49152": {
+      "content": "<|start_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49153": {
+      "content": "<|end_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49154": {
+      "content": "<|tool_call|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "None",
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>"
+  ],
+  "bos_token": "<|end_of_text|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|end_of_text|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 8192,
+  "pad_token": "<|end_of_text|>",
+  "padding_side": "left",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|end_of_text|>",
+  "vocab_size": 49152
+}