--- license: apache-2.0 library_name: transformers base_model: Qwen/Qwen3.5-4B-Base tags: - dpo - safety-tuning --- # Qwen3.5-4B-Base DPO at step 43 Checkpoint after 43 optimizer updates (0.5 epochs) of direct preference optimization. For each prompt, both responses came from Qwen3.8-27B. Binary score 1 supplied the chosen response and score 0 the rejected response. The eight-dataset training mixture contains 5,493 preference pairs across 1,755 prompts. The run started from Qwen/Qwen3.5-4B-Base at revision `1001bb4d826a52d1f399e183466143f4da7b741b`. Training uses beta 0.1, batch size 64 pairs and peak learning rate 1e-6. The export includes the trained language weights and preserves the base model's vision and MTP weights.