Commit
·
5243887
1
Parent(s):
94cd439
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ This is a LoRA for Llama 2 13B trained on a translated [alpaca dataset](https://
|
|
| 13 |
Base model used was [The Bloke's Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) trained in 4bit precision with an added padding token.
|
| 14 |
|
| 15 |
## Important INFO
|
| 16 |
-
The original Llama 2 model does not have a padding token, this came to be restrictive
|
| 17 |
```python
|
| 18 |
from transformers import LlamaTokenizer, LlamaForCausalLM
|
| 19 |
|
|
|
|
| 13 |
Base model used was [The Bloke's Llama-2-13B-fp16](https://huggingface.co/TheBloke/Llama-2-13B-fp16) trained in 4bit precision with an added padding token.
|
| 14 |
|
| 15 |
## Important INFO
|
| 16 |
+
The original Llama 2 model does not have a padding token, this came to be restrictive when training. To address this, I added a padding token to the tokenizer associated with the model.
|
| 17 |
```python
|
| 18 |
from transformers import LlamaTokenizer, LlamaForCausalLM
|
| 19 |
|