johnsmith968530 commited on
Commit
cc1fbd6
·
verified ·
1 Parent(s): 6c58ba4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -20
README.md CHANGED
@@ -18,24 +18,20 @@ This model [johnsmith968530/WeiboAI-VibeThinker-1.5B-MLX-8bit](https://huggingfa
18
  converted to MLX format from [WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B)
19
  using mlx-lm version **0.28.3**.
20
 
21
- ## Use with mlx
22
-
23
  ```bash
24
- pip install mlx-lm
25
- ```
26
-
27
- ```python
28
- from mlx_lm import load, generate
29
-
30
- model, tokenizer = load("johnsmith968530/WeiboAI-VibeThinker-1.5B-MLX-8bit")
31
-
32
- prompt = "hello"
33
-
34
- if tokenizer.chat_template is not None:
35
- messages = [{"role": "user", "content": prompt}]
36
- prompt = tokenizer.apply_chat_template(
37
- messages, add_generation_prompt=True
38
- )
39
-
40
- response = generate(model, tokenizer, prompt=prompt, verbose=True)
41
- ```
 
18
  converted to MLX format from [WeiboAI/VibeThinker-1.5B](https://huggingface.co/WeiboAI/VibeThinker-1.5B)
19
  using mlx-lm version **0.28.3**.
20
 
 
 
21
  ```bash
22
+ export MODEL1_MAJOR="WeiboAI"
23
+ export MODEL1_MINOR="VibeThinker-1.5B"
24
+ export MODEL1_Q_BITS="8"
25
+ export MODEL1_SD="$(stardate)"
26
+
27
+ echodo () { echo "$@" && "$@"; }
28
+ mkdir -p /tmp/mlx_lm/convert
29
+
30
+ echodo time mlx_lm.convert \
31
+ --hf-path "$MODEL1_MAJOR/$MODEL1_MINOR" \
32
+ --mlx-path "/tmp/mlx_lm/convert/$MODEL1_SD" \
33
+ --quantize \
34
+ --q-bits "$MODEL1_Q_BITS" \
35
+ --upload-repo \
36
+ "johnsmith968530/$MODEL1_MAJOR-$MODEL1_MINOR-MLX-${MODEL1_Q_BITS}bit"
37
+ ```