koelectra-samll-klue-mrc

This model is a fine-tuned version of monologg/koelectra-small-discriminator on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6956

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	3	1.8687
No log	2.0	6	1.8565
No log	3.0	9	1.8483
No log	4.0	12	1.8408
No log	5.0	15	1.8276
No log	6.0	18	1.8200
No log	7.0	21	1.8148
No log	8.0	24	1.8055
No log	9.0	27	1.7952
No log	10.0	30	1.7888
No log	11.0	33	1.7829
No log	12.0	36	1.7767
No log	13.0	39	1.7709
No log	14.0	42	1.7642
No log	15.0	45	1.7585
No log	16.0	48	1.7538
No log	17.0	51	1.7495
No log	18.0	54	1.7446
No log	19.0	57	1.7393
No log	20.0	60	1.7357
No log	21.0	63	1.7321
No log	22.0	66	1.7276
No log	23.0	69	1.7239
No log	24.0	72	1.7208
No log	25.0	75	1.7178
No log	26.0	78	1.7149
No log	27.0	81	1.7126
No log	28.0	84	1.7100
No log	29.0	87	1.7077
No log	30.0	90	1.7060
No log	31.0	93	1.7047
No log	32.0	96	1.7029
No log	33.0	99	1.7010
No log	34.0	102	1.6994
No log	35.0	105	1.6982
No log	36.0	108	1.6973
No log	37.0	111	1.6967
No log	38.0	114	1.6961
No log	39.0	117	1.6957
No log	40.0	120	1.6956

Framework versions

• Transformers 4.53.2
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.2