Qwen3.5-4B-Base OPD โ€” Mixture8, step 5

Research checkpoint after 5 completed optimizer updates of on-policy distillation (OPD). The source Megatron iteration is zero-based 4 (iter_0000004).

Setting Value
Initialization Qwen/Qwen3.5-4B-Base, revision 1001bb4d826a52d1f399e183466143f4da7b741b
Teacher Qwen/Qwen3.8-27B, revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
Objective Plain reverse-KL OPD, no additional reference KL
Prompt pool Latest eight-dataset mixture, preserved source prompt order
Learning rate Constant 1e-6
Batch 30 prompts ร— 4 on-policy responses = 120
Generation / sequence limits 32,768 / 65,536 tokens
Sampling Temperature 1.0, seed 42
Saved 2026-11-18, first five-update boundary

Teacher requests use the archived dataset-specific generation system: policy Spec on harmful/difficult prompts and no added Spec on benign prompts. This is the Base-initialized arm, distinct from the queued SFT-step121 and SFT-step364 initializations. Training run: opd-q38t-base-lr1e6-20steps-save5-20261118-v1.

The repository contains Hugging Face inference weights and tokenizer/configuration assets. Trained language weights were converted from the committed distributed checkpoint, then verified by exact tensor round-trip using the project's inverse Qwen3.5 mapping. Vision/MTP weights absent from the text-only training checkpoint are retained unchanged from the pinned base model. Optimizer/RNG state and training data are not included.

This early checkpoint has not been benchmark-evaluated; no safety or quality improvement is claimed. It retains the base model's tokenizer and architecture and requires a Transformers version supporting Qwen3.5.

Downloads last month
-
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for thkim0305-SNU/Safety_Tuning-Qwen3.5-4B-Base-OPD-Mixture8-Qwen3.8-27B-lr1e-6-step5

Finetuned
(309)
this model