mobicham commited on
Commit
9d52878
·
verified ·
1 Parent(s): 6283cc7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -2
README.md CHANGED
@@ -7,9 +7,15 @@ pipeline_tag: text-generation
7
  ## Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ
8
  This is a version of the Mixtral-8x7B-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) quantized to 4-bit via Half-Quadratic Quantization (HQQ).
9
  ### Basic Usage
10
- To run the model, install the HQQ library from https://github.com/mobiusml/hqq and use it as follows:
 
 
 
 
 
 
11
  ``` Python
12
- model_id = 'mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ/'
13
 
14
  #Load the model
15
  from hqq.engine.hf import HQQModelForCausalLM, AutoTokenizer
 
7
  ## Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ
8
  This is a version of the Mixtral-8x7B-v0.1 model (https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) quantized to 4-bit via Half-Quadratic Quantization (HQQ).
9
  ### Basic Usage
10
+ To run the model, install the HQQ library:
11
+ ```
12
+ pip install hqq==0.1.8
13
+ pip install transformers==4.46.0
14
+ ```
15
+
16
+ and use it as follows:
17
  ``` Python
18
+ model_id = 'mobiuslabsgmbh/Mixtral-8x7B-v0.1-hf-4bit_g64-HQQ'
19
 
20
  #Load the model
21
  from hqq.engine.hf import HQQModelForCausalLM, AutoTokenizer