Text Generation
GGUF
English
quant
experimental
eaddario commited on
Commit
8f92747
·
verified ·
1 Parent(s): 897df0d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -15
README.md CHANGED
@@ -116,21 +116,21 @@ For the test data used in the generation of these scores, follow the appropiate
116
 
117
  | Model | ARC | HellaSwag | MMLU | Truthful QA | WinoGrande | Avg Score |
118
  | ----------------------------------------------------------------------------------------------- | --------------: | --------: | --------------: | --------------: | --------------: | --------: |
119
- | [Mistral-Small-3.2-24B-Instruct-2506-IQ3_M](./Mistral-Small-3.2-24B-Instruct-2506-IQ3_M.gguf) | 69.7333 ±1.6787 | 83.87 | 43.4667 ±1.8113 | 34.9333 ±1.7420 | 78.0000 ±1.5136 | 56.40 |
120
- | [Mistral-Small-3.2-24B-Instruct-2506-IQ3_S](./Mistral-Small-3.2-24B-Instruct-2506-IQ3_S.gguf) | 67.6000 ±1.7100 | 83.07 | 44.1333 ±1.8143 | 35.4667 ±1.7481 | 77.6000 ±1.5234 | 56.67 |
121
- | [Mistral-Small-3.2-24B-Instruct-2506-IQ4_NL](./Mistral-Small-3.2-24B-Instruct-2506-IQ4_NL.gguf) | 69.2000 ±1.6869 | 84.93 | 43.0667 ±1.8093 | 37.8667 ±1.7724 | 78.9333 ±1.4900 | 57.47 |
122
- | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_L](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_L.gguf) | 69.0667 ±1.6889 | 82.27 | 43.3333 ±1.8106 | 35.4667 ±1.7481 | 78.0000 ±1.5136 | 57.25 |
123
- | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_M.gguf) | 68.9333 ±1.6909 | 82.00 | 43.2000 ±1.8100 | 36.6667 ±1.7608 | 78.1333 ±1.5103 | 57.76 |
124
- | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_S.gguf) | 69.8667 ±1.6766 | 81.46 | 42.8000 ±1.8079 | 34.5333 ±1.7374 | 76.1333 ±1.5576 | 55.39 |
125
- | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf) | 69.0667 ±1.6889 | 83.33 | 45.0667 ±1.8180 | 37.4667 ±1.7686 | 79.0667 ±1.4865 | 56.61 |
126
- | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M-bartowski][b-q4km] | 67.0667 ±1.7172 | 83.60 | 45.7333 ±1.8203 | 36.2667 ±1.7567 | 79.4667 ±1.4760 | 61.73 |
127
- | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M-unsloth][u-q4km] | 66.6667 ±1.7225 | 84.00 | 45.8667 ±1.8207 | 35.7333 ±1.7510 | 79.3333 ±1.4795 | 61.92 |
128
- | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q4_K_S.gguf) | 69.2000 ±1.6869 | 82.93 | 44.5333 ±1.8160 | 35.8667 ±1.7525 | 78.8000 ±1.4934 | 57.31 |
129
- | [Mistral-Small-3.2-24B-Instruct-2506-Q5_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q5_K_M.gguf) | 68.9333 ±1.6909 | 84.13 | 45.7333 ±1.8203 | 35.7333 ±1.7510 | 79.7333 ±1.4688 | 57.36 |
130
- | [Mistral-Small-3.2-24B-Instruct-2506-Q5_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q5_K_S.gguf) | 69.2000 ±1.6869 | 84.13 | 45.2000 ±1.8185 | 36.0000 ±1.7539 | 78.9333 ±1.4900 | 57.17 |
131
- | [Mistral-Small-3.2-24B-Instruct-2506-Q6_K](./Mistral-Small-3.2-24B-Instruct-2506-Q6_K.gguf) | 68.0000 ±1.7045 | 84.26 | 44.5333 ±1.8160 | 36.2667 ±1.7567 | 79.2000 ±1.4830 | 57.41 |
132
- | [Mistral-Small-3.2-24B-Instruct-2506-Q8_0](./Mistral-Small-3.2-24B-Instruct-2506-Q8_0.gguf) | 68.4000 ±1.6988 | 84.27 | 44.5333 ±1.8160 | 36.8000 ±1.7621 | 79.0667 ±1.4865 | 57.28 |
133
- | [Mistral-Small-3.2-24B-Instruct-2506-F16](./Mistral-Small-3.2-24B-Instruct-2506-F16.gguf) | 68.2667 ±1.7007 | 84.40 | 45.2000 ±1.8185 | 36.8000 ±1.7621 | 79.4667 ±1.4760 | 61.95 |
134
 
135
  ### Tokens per Second - Benchmarks
136
  Scores generated using [llama-bench][bch]. Naive (`llama-quantize` with no optimization) Q4_K_M quantization included for comparison.
 
116
 
117
  | Model | ARC | HellaSwag | MMLU | Truthful QA | WinoGrande | Avg Score |
118
  | ----------------------------------------------------------------------------------------------- | --------------: | --------: | --------------: | --------------: | --------------: | --------: |
119
+ | [Mistral-Small-3.2-24B-Instruct-2506-IQ3_M](./Mistral-Small-3.2-24B-Instruct-2506-IQ3_M.gguf) | 69.7333 ±1.6787 | 83.87 | 43.4667 ±1.8113 | 34.9333 ±1.7420 | 78.0000 ±1.5136 | 62.00 |
120
+ | [Mistral-Small-3.2-24B-Instruct-2506-IQ3_S](./Mistral-Small-3.2-24B-Instruct-2506-IQ3_S.gguf) | 67.6000 ±1.7100 | 83.07 | 44.1333 ±1.8143 | 35.4667 ±1.7481 | 77.6000 ±1.5234 | 61.57 |
121
+ | [Mistral-Small-3.2-24B-Instruct-2506-IQ4_NL](./Mistral-Small-3.2-24B-Instruct-2506-IQ4_NL.gguf) | 69.2000 ±1.6869 | 84.93 | 43.0667 ±1.8093 | 37.8667 ±1.7724 | 78.9333 ±1.4900 | 62.80 |
122
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_L](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_L.gguf) | 69.0667 ±1.6889 | 82.27 | 43.3333 ±1.8106 | 35.4667 ±1.7481 | 78.0000 ±1.5136 | 61.63 |
123
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_M.gguf) | 68.9333 ±1.6909 | 82.00 | 43.2000 ±1.8100 | 36.6667 ±1.7608 | 78.1333 ±1.5103 | 61.79 |
124
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q3_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q3_K_S.gguf) | 69.8667 ±1.6766 | 81.46 | 42.8000 ±1.8079 | 34.5333 ±1.7374 | 76.1333 ±1.5576 | 60.96 |
125
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M.gguf) | 69.0667 ±1.6889 | 83.33 | 45.0667 ±1.8180 | 37.4667 ±1.7686 | 79.0667 ±1.4865 | 62.80 |
126
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M-bartowski][b-q4km] | 67.0667 ±1.7172 | 83.60 | 45.7333 ±1.8203 | 36.2667 ±1.7567 | 79.4667 ±1.4760 | 62.43 |
127
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_M-unsloth][u-q4km] | 66.6667 ±1.7225 | 84.00 | 45.8667 ±1.8207 | 35.7333 ±1.7510 | 79.3333 ±1.4795 | 62.32 |
128
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q4_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q4_K_S.gguf) | 69.2000 ±1.6869 | 82.93 | 44.5333 ±1.8160 | 35.8667 ±1.7525 | 78.8000 ±1.4934 | 62.27 |
129
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q5_K_M](./Mistral-Small-3.2-24B-Instruct-2506-Q5_K_M.gguf) | 68.9333 ±1.6909 | 84.13 | 45.7333 ±1.8203 | 35.7333 ±1.7510 | 79.7333 ±1.4688 | 62.85 |
130
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q5_K_S](./Mistral-Small-3.2-24B-Instruct-2506-Q5_K_S.gguf) | 69.2000 ±1.6869 | 84.13 | 45.2000 ±1.8185 | 36.0000 ±1.7539 | 78.9333 ±1.4900 | 62.69 |
131
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q6_K](./Mistral-Small-3.2-24B-Instruct-2506-Q6_K.gguf) | 68.0000 ±1.7045 | 84.26 | 44.5333 ±1.8160 | 36.2667 ±1.7567 | 79.2000 ±1.4830 | 62.45 |
132
+ | [Mistral-Small-3.2-24B-Instruct-2506-Q8_0](./Mistral-Small-3.2-24B-Instruct-2506-Q8_0.gguf) | 68.4000 ±1.6988 | 84.27 | 44.5333 ±1.8160 | 36.8000 ±1.7621 | 79.0667 ±1.4865 | 62.61 |
133
+ | [Mistral-Small-3.2-24B-Instruct-2506-F16](./Mistral-Small-3.2-24B-Instruct-2506-F16.gguf) | 68.2667 ±1.7007 | 84.40 | 45.2000 ±1.8185 | 36.8000 ±1.7621 | 79.4667 ±1.4760 | 62.83 |
134
 
135
  ### Tokens per Second - Benchmarks
136
  Scores generated using [llama-bench][bch]. Naive (`llama-quantize` with no optimization) Q4_K_M quantization included for comparison.