Hugging Face's logo

yangzhang33
/
scibert_scivocab_uncased-finetuned-molstm-lpm-0.3-25epochs

Fill-Mask
Transformers
Safetensors
bert
Generated from Trainer
Model card Files
xet
Community
zy commited on
Commit
941d272
·
verified ·
1 Parent(s): 7a71015

End of training

Browse files
Files changed (3) hide show
  1. README.md +81 -0
  2. generation_config.json +5 -0
  3. model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: allenai/scibert_scivocab_uncased
3
+ tags:
4
+ - generated_from_trainer
5
+ model-index:
6
+ - name: scibert_scivocab_uncased-finetuned-molstm-lpm-0.3-25epochs
7
+ results: []
8
+ ---
9
+
10
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
11
+ should probably proofread and complete it, then remove this comment. -->
12
+
13
+ # scibert_scivocab_uncased-finetuned-molstm-lpm-0.3-25epochs
14
+
15
+ This model is a fine-tuned version of [allenai/scibert_scivocab_uncased](https://huggingface.co/allenai/scibert_scivocab_uncased) on an unknown dataset.
16
+ It achieves the following results on the evaluation set:
17
+ - Loss: 0.0407
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+ ### Training hyperparameters
34
+
35
+ The following hyperparameters were used during training:
36
+ - learning_rate: 2e-05
37
+ - train_batch_size: 32
38
+ - eval_batch_size: 32
39
+ - seed: 42
40
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
41
+ - lr_scheduler_type: linear
42
+ - num_epochs: 25
43
+ - mixed_precision_training: Native AMP
44
+
45
+ ### Training results
46
+
47
+ | Training Loss | Epoch | Step | Validation Loss |
48
+ |:-------------:|:-----:|:-----:|:---------------:|
49
+ | 0.1073 | 1.0 | 3301 | 0.0633 |
50
+ | 0.0657 | 2.0 | 6602 | 0.0563 |
51
+ | 0.059 | 3.0 | 9903 | 0.0536 |
52
+ | 0.0556 | 4.0 | 13204 | 0.0518 |
53
+ | 0.0531 | 5.0 | 16505 | 0.0496 |
54
+ | 0.0511 | 6.0 | 19806 | 0.0489 |
55
+ | 0.0498 | 7.0 | 23107 | 0.0477 |
56
+ | 0.0488 | 8.0 | 26408 | 0.0468 |
57
+ | 0.0478 | 9.0 | 29709 | 0.0464 |
58
+ | 0.0467 | 10.0 | 33010 | 0.0455 |
59
+ | 0.0467 | 11.0 | 36311 | 0.0450 |
60
+ | 0.0458 | 12.0 | 39612 | 0.0454 |
61
+ | 0.0449 | 13.0 | 42913 | 0.0441 |
62
+ | 0.0447 | 14.0 | 46214 | 0.0432 |
63
+ | 0.044 | 15.0 | 49515 | 0.0428 |
64
+ | 0.0436 | 16.0 | 52816 | 0.0429 |
65
+ | 0.0433 | 17.0 | 56117 | 0.0428 |
66
+ | 0.0431 | 18.0 | 59418 | 0.0423 |
67
+ | 0.0427 | 19.0 | 62719 | 0.0419 |
68
+ | 0.0425 | 20.0 | 66020 | 0.0420 |
69
+ | 0.0422 | 21.0 | 69321 | 0.0412 |
70
+ | 0.0422 | 22.0 | 72622 | 0.0413 |
71
+ | 0.0416 | 23.0 | 75923 | 0.0407 |
72
+ | 0.0415 | 24.0 | 79224 | 0.0410 |
73
+ | 0.0411 | 25.0 | 82525 | 0.0408 |
74
+
75
+
76
+ ### Framework versions
77
+
78
+ - Transformers 4.38.2
79
+ - Pytorch 2.0.1
80
+ - Datasets 2.18.0
81
+ - Tokenizers 0.15.2
generation_config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "pad_token_id": 0,
4
+ "transformers_version": "4.38.2"
5
+ }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e6b971df1592c8e98b96b2c00d710de2c0310d6b91c32847946ea8ff3514a3ec
3
  size 439828064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5e94cbc45f00f4d2f3e9aeb91ce10b6d2a17cc6c7e2fbddc08ae5f4486bc4d7e
3
  size 439828064