How to use from
Docker Model Runner
docker model run hf.co/thkim0305-SNU/Safety_Tuning-Qwen3.5-4B-Base-DPO-Mixture8-Teacher1-Teacher0-lr1e-6-step43
Quick Links

Qwen3.5-4B-Base DPO at step 43

Checkpoint after 43 optimizer updates (0.5 epochs) of direct preference optimization. For each prompt, both responses came from Qwen3.8-27B. Binary score 1 supplied the chosen response and score 0 the rejected response. The eight-dataset training mixture contains 5,493 preference pairs across 1,755 prompts.

The run started from Qwen/Qwen3.5-4B-Base at revision 1001bb4d826a52d1f399e183466143f4da7b741b. Training uses beta 0.1, batch size 64 pairs and peak learning rate 1e-6. The export includes the trained language weights and preserves the base model's vision and MTP weights.

Downloads last month
17
Safetensors
Model size
5B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for thkim0305-SNU/Safety_Tuning-Qwen3.5-4B-Base-DPO-Mixture8-Teacher1-Teacher0-lr1e-6-step43

Finetuned
(323)
this model