koelectra-small-clue-mrc

This model is a fine-tuned version of monologg/koelectra-small-discriminator on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 7.1824

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	10	6.1555
No log	2.0	20	6.2147
No log	3.0	30	6.2340
No log	4.0	40	6.2524
No log	5.0	50	6.3273
No log	6.0	60	6.3144
No log	7.0	70	6.3801
No log	8.0	80	6.3412
No log	9.0	90	6.3442
No log	10.0	100	6.3031
No log	11.0	110	6.4105
No log	12.0	120	6.3805
No log	13.0	130	6.2836
No log	14.0	140	6.3430
No log	15.0	150	6.4388
No log	16.0	160	6.4083
No log	17.0	170	6.5182
No log	18.0	180	6.4754
No log	19.0	190	6.5464
No log	20.0	200	6.5799
No log	21.0	210	6.5363
No log	22.0	220	6.5673
No log	23.0	230	6.4947
No log	24.0	240	6.5463
No log	25.0	250	6.4552
No log	26.0	260	6.6232
No log	27.0	270	6.5952
No log	28.0	280	6.6527
No log	29.0	290	6.7180
No log	30.0	300	6.7968
No log	31.0	310	6.7918
No log	32.0	320	6.6616
No log	33.0	330	6.7570
No log	34.0	340	6.7317
No log	35.0	350	6.7470
No log	36.0	360	6.7407
No log	37.0	370	6.7294
No log	38.0	380	6.6954
No log	39.0	390	6.7345
No log	40.0	400	6.6893
No log	41.0	410	6.7154
No log	42.0	420	6.7860
No log	43.0	430	6.7898
No log	44.0	440	6.7796
No log	45.0	450	6.8280
No log	46.0	460	6.7464
No log	47.0	470	6.9669
No log	48.0	480	6.8378
No log	49.0	490	6.8529
0.8074	50.0	500	7.0133
0.8074	51.0	510	7.0422
0.8074	52.0	520	6.9611
0.8074	53.0	530	6.9869
0.8074	54.0	540	6.9359
0.8074	55.0	550	6.9736
0.8074	56.0	560	7.0233
0.8074	57.0	570	6.8595
0.8074	58.0	580	6.9922
0.8074	59.0	590	6.9933
0.8074	60.0	600	6.9497
0.8074	61.0	610	6.9149
0.8074	62.0	620	6.9843
0.8074	63.0	630	6.9076
0.8074	64.0	640	6.9961
0.8074	65.0	650	7.0133
0.8074	66.0	660	7.0827
0.8074	67.0	670	7.0696
0.8074	68.0	680	7.1281
0.8074	69.0	690	7.0869
0.8074	70.0	700	6.9208
0.8074	71.0	710	7.0654
0.8074	72.0	720	7.1179
0.8074	73.0	730	7.0358
0.8074	74.0	740	7.0207
0.8074	75.0	750	7.0764
0.8074	76.0	760	7.0927
0.8074	77.0	770	7.1204
0.8074	78.0	780	7.1306
0.8074	79.0	790	7.0395
0.8074	80.0	800	7.0904
0.8074	81.0	810	7.1728
0.8074	82.0	820	7.1736
0.8074	83.0	830	7.1737
0.8074	84.0	840	7.1170
0.8074	85.0	850	7.1273
0.8074	86.0	860	7.1002
0.8074	87.0	870	7.1119
0.8074	88.0	880	7.1539
0.8074	89.0	890	7.1715
0.8074	90.0	900	7.1523
0.8074	91.0	910	7.1583
0.8074	92.0	920	7.1967
0.8074	93.0	930	7.2076
0.8074	94.0	940	7.1889
0.8074	95.0	950	7.1646
0.8074	96.0	960	7.1693
0.8074	97.0	970	7.1801
0.8074	98.0	980	7.1830
0.8074	99.0	990	7.1843
0.4709	100.0	1000	7.1824

Framework versions

• Transformers 4.53.2
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.2