feihu.hf
commited on
Commit
·
6329251
1
Parent(s):
34778e2
update README
Browse files
README.md
CHANGED
|
@@ -30,7 +30,7 @@ Qwen3 is the latest generation of large language models in Qwen series, offering
|
|
| 30 |
- Number of Paramaters (Non-Embedding): 3.6B
|
| 31 |
- Number of Layers: 36
|
| 32 |
- Number of Attention Heads (GQA): 32 for Q and 8 for KV
|
| 33 |
-
- Context Length: 32,768 natively and [131,072 tokens with YaRN](#processing-long-texts).
|
| 34 |
|
| 35 |
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0
|
| 36 |
|
|
@@ -46,7 +46,7 @@ We advise you to clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp) and
|
|
| 46 |
In the following demonstration, we assume that you are running commands under the repository `llama.cpp`.
|
| 47 |
|
| 48 |
```shell
|
| 49 |
-
./llama-cli -hf Qwen/Qwen3-4B:Q8_0 --jinja --color -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 1.5 -c 40960 -n 32768 --no-context-shift
|
| 50 |
```
|
| 51 |
|
| 52 |
### ollama
|
|
|
|
| 30 |
- Number of Paramaters (Non-Embedding): 3.6B
|
| 31 |
- Number of Layers: 36
|
| 32 |
- Number of Attention Heads (GQA): 32 for Q and 8 for KV
|
| 33 |
+
- Context Length: 32,768 natively and [131,072 tokens with YaRN](#processing-long-texts).
|
| 34 |
|
| 35 |
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0
|
| 36 |
|
|
|
|
| 46 |
In the following demonstration, we assume that you are running commands under the repository `llama.cpp`.
|
| 47 |
|
| 48 |
```shell
|
| 49 |
+
./llama-cli -hf Qwen/Qwen3-4B-GGUF:Q8_0 --jinja --color -ngl 99 -fa -sm row --temp 0.6 --top-k 20 --top-p 0.95 --min-p 0 --presence-penalty 1.5 -c 40960 -n 32768 --no-context-shift
|
| 50 |
```
|
| 51 |
|
| 52 |
### ollama
|
params
CHANGED
|
@@ -9,5 +9,6 @@
|
|
| 9 |
"presence_penalty" : 1.5,
|
| 10 |
"top_k" : 20,
|
| 11 |
"top_p" : 0.95,
|
| 12 |
-
"num_predict" : 32768
|
|
|
|
| 13 |
}
|
|
|
|
| 9 |
"presence_penalty" : 1.5,
|
| 10 |
"top_k" : 20,
|
| 11 |
"top_p" : 0.95,
|
| 12 |
+
"num_predict" : 32768,
|
| 13 |
+
"num_ctx": 40960
|
| 14 |
}
|