Commit
·
8359b19
1
Parent(s):
7dfebcc
Update README.md
Browse files
README.md
CHANGED
|
@@ -18,16 +18,16 @@ Of note, the base checkpoint used was from commit "final model" fad4f1a5cd0563ac
|
|
| 18 |
|
| 19 |
### Wikitext (wikitext-2-raw-v1_train) Perplexity (64 rows) as evaluated via [exllamav2](https://github.com/turboderp/exllamav2):
|
| 20 |
|
| 21 |
-
| Model | 2048
|
| 22 |
-
| ---------------------- |
|
| 23 |
-
| TinyLlama-1.1B | 8.5633 | 208.3586
|
| 24 |
-
| TinyLlama-1.1B-32k
|
| 25 |
|
| 26 |
### Evaluation on HumanEval by [turboderp](https://huggingface.co/turboderp):
|
| 27 |
|
| 28 |
-
| Model | Pass@1
|
| 29 |
-
| -------------------------------------- |
|
| 30 |
-
| TinyLlama-1.1B | 0.0841 | 0.1524
|
| 31 |
-
| TinyLlama-1.1B (NTK alpha=7.7) | 0.0598
|
| 32 |
-
| TinyLlama-1.1B-32k-ckpt-554 | 0.0732
|
| 33 |
-
| TinyLlama-1.1B-32k
|
|
|
|
| 18 |
|
| 19 |
### Wikitext (wikitext-2-raw-v1_train) Perplexity (64 rows) as evaluated via [exllamav2](https://github.com/turboderp/exllamav2):
|
| 20 |
|
| 21 |
+
| Model | 2048 | 4096 | 8192 | 16384 | 32768 |
|
| 22 |
+
| ---------------------- | ---------- | ---------- | ---------- | ---------- | ---------- |
|
| 23 |
+
| TinyLlama-1.1B | **8.5633** | 208.3586 | 863.7507 | 1600.5021 | 6981.9021 |
|
| 24 |
+
| **TinyLlama-1.1B-32k** | 8.6548 | **7.8339** | **7.4904** | **7.3674** | **7.1338** |
|
| 25 |
|
| 26 |
### Evaluation on HumanEval by [turboderp](https://huggingface.co/turboderp):
|
| 27 |
|
| 28 |
+
| Model | Pass@1 | Pass@10 |
|
| 29 |
+
| -------------------------------------- | --------------- | ----------- |
|
| 30 |
+
| TinyLlama-1.1B | **0.0841** | **0.1524** |
|
| 31 |
+
| TinyLlama-1.1B (NTK alpha=7.7) | 0.0598 | 0.1098 |
|
| 32 |
+
| TinyLlama-1.1B-32k-ckpt-554 | 0.0732 | 0.1402 |
|
| 33 |
+
| **TinyLlama-1.1B-32k** | 0.0829 | **0.1524** |
|