qwen14b-multi-turn-R3

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-14B on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2351

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.6453	0.1647	200	0.6485
0.5694	0.3295	400	0.5748
0.5234	0.4942	600	0.5111
0.4379	0.6590	800	0.4559
0.4436	0.8237	1000	0.4048
0.404	0.9885	1200	0.3584
0.2826	1.1532	1400	0.3257
0.2371	1.3180	1600	0.3000
0.2523	1.4827	1800	0.2749
0.1946	1.6474	2000	0.2569
0.1662	1.8122	2200	0.2425
0.1907	1.9769	2400	0.2351

Framework versions

• PEFT 0.17.1
• Transformers 4.56.2
• Pytorch 2.6.0+cu124
• Datasets 4.1.1
• Tokenizers 0.22.1