Upload README.md
Browse files
README.md
CHANGED
|
@@ -350,13 +350,16 @@ And thank you again to a16z for their generous grant.
|
|
| 350 |
|
| 351 |
# NeuralHermes 2.5 - Mistral 7B
|
| 352 |
|
| 353 |
-
NeuralHermes is an [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) model that has been further fine-tuned with Direct Preference Optimization (DPO) using the [mlabonne/chatml_dpo_pairs](https://huggingface.co/datasets/mlabonne/chatml_dpo_pairs) dataset. It surpasses the original model on several benchmarks (see results)
|
| 354 |
|
| 355 |
It is directly inspired by the RLHF process described by [Intel/neural-chat-7b-v3-1](https://huggingface.co/Intel/neural-chat-7b-v3-1)'s authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.
|
| 356 |
|
| 357 |
The code to train this model is available on [Google Colab](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing) and [GitHub](https://github.com/mlabonne/llm-course/tree/main). It required an A100 GPU for about an hour.
|
| 358 |
|
| 359 |
-
|
|
|
|
|
|
|
|
|
|
| 360 |
|
| 361 |
## Results
|
| 362 |
|
|
|
|
| 350 |
|
| 351 |
# NeuralHermes 2.5 - Mistral 7B
|
| 352 |
|
| 353 |
+
NeuralHermes is an [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) model that has been further fine-tuned with Direct Preference Optimization (DPO) using the [mlabonne/chatml_dpo_pairs](https://huggingface.co/datasets/mlabonne/chatml_dpo_pairs) dataset. It surpasses the original model on several benchmarks (see results).
|
| 354 |
|
| 355 |
It is directly inspired by the RLHF process described by [Intel/neural-chat-7b-v3-1](https://huggingface.co/Intel/neural-chat-7b-v3-1)'s authors to improve performance. I used the same dataset and reformatted it to apply the ChatML template.
|
| 356 |
|
| 357 |
The code to train this model is available on [Google Colab](https://colab.research.google.com/drive/15iFBr1xWgztXvhrj5I9fBv20c7CFOPBE?usp=sharing) and [GitHub](https://github.com/mlabonne/llm-course/tree/main). It required an A100 GPU for about an hour.
|
| 358 |
|
| 359 |
+
### Quantized models
|
| 360 |
+
* GGUF: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-GGUF
|
| 361 |
+
* AWQ: https://huggingface.co/TheBloke/NeuralHermes-2.5-Mistral-7B-AWQ
|
| 362 |
+
* EXL2 (5pbw): https://huggingface.co/IconicAI/NeuralHermes-2.5-Mistral-7B-exl2-5bpw
|
| 363 |
|
| 364 |
## Results
|
| 365 |
|