ENOSYS/Qwen3.5-2B-heretic-250-v1-GGUF

Experimental global target bits‑per‑weight quantization of coder3101/Qwen3.5-2B-heretic

• Using non-standard (forked) LLaMA C++ branch for quantization.
• Using a CLI tool to build KLD evaluation and imatrix calibration datasets for GGUF models, sourced from eaddario/imatrix-calibration.
• Using dataset sources: tools, text_en, text_ru.
• Using dataset chunks: 250.
• Tensors quantinization F16 instead of BF16, Nvidia Pascal architecture friendly like P100.
• Small set of patches added.
• Multimodal work perfectly, use:
  • mmproj-F16
  • mmproj-F32

Many thanks to Ed Addario for an impressive job.

Quantization comparison

BPW	PPL correlation	PPL mean ratio	ΔPPL	Mean KLD	Maximum KLD	99.9% KLD	Mean Δp	RMS Δp
5.00	99.64%	1.026057 ± 0.001098	0.571999 ± 0.024867	0.028004 ± 0.000171	2.310003	0.489172	-0.530 ± 0.016 %	4.045 ± 0.043 %
5.25	99.70%	1.022321 ± 0.001001	0.489982 ± 0.022620	0.023515 ± 0.000153	3.049644	0.418944	-0.442 ± 0.015 %	3.740 ± 0.044 %
5.30	99.70%	1.021006 ± 0.000993	0.461111 ± 0.022363	0.022725 ± 0.000146	3.486933	0.401278	-0.425 ± 0.015 %	3.711 ± 0.042 %
5.50	99.78%	1.018126 ± 0.000842	0.397896 ± 0.019117	0.016384 ± 0.000109	2.211204	0.266915	-0.311 ± 0.013 %	3.141 ± 0.036 %
5.75	99.85%	1.012000 ± 0.000695	0.263430 ± 0.015510	0.011618 ± 0.000070	2.060406	0.177300	-0.278 ± 0.011 %	2.707 ± 0.029 %
5.80	99.82%	1.015834 ± 0.000776	0.347594 ± 0.017563	0.013770 ± 0.000105	2.561136	0.252383	-0.256 ± 0.012 %	2.901 ± 0.039 %
6.00	99.87%	1.011607 ± 0.000641	0.254793 ± 0.014392	0.009732 ± 0.000070	2.332554	0.151923	-0.210 ± 0.010 %	2.452 ± 0.035 %
6.25	99.89%	1.011164 ± 0.000589	0.245080 ± 0.013269	0.007907 ± 0.000069	2.916830	0.129800	-0.203 ± 0.009 %	2.190 ± 0.033 %
6.30	99.90%	1.011153 ± 0.000577	0.244824 ± 0.013017	0.007673 ± 0.000058	2.056196	0.129539	-0.202 ± 0.009 %	2.161 ± 0.031 %
6.50	99.93%	1.007030 ± 0.000487	0.154329 ± 0.010840	0.005530 ± 0.000034	0.793070	0.086250	-0.156 ± 0.007 %	1.828 ± 0.025 %
6.75	99.95%	1.005130 ± 0.000400	0.112623 ± 0.008912	0.003585 ± 0.000023	0.638760	0.058130	-0.082 ± 0.006 %	1.471 ± 0.020 %
6.80	99.95%	1.004828 ± 0.000397	0.105980 ± 0.008823	0.003551 ± 0.000022	0.470148	0.056400	-0.081 ± 0.006 %	1.475 ± 0.021 %
7.00	99.95%	1.003980 ± 0.000380	0.087374 ± 0.008441	0.003187 ± 0.000019	0.355283	0.045939	-0.048 ± 0.006 %	1.381 ± 0.017 %
7.25	99.96%	1.002210 ± 0.000353	0.048507 ± 0.007781	0.002755 ± 0.000014	0.204998	0.040443	-0.056 ± 0.005 %	1.275 ± 0.013 %
7.30	99.96%	1.002463 ± 0.000351	0.054060 ± 0.007734	0.002745 ± 0.000016	0.321245	0.044340	-0.054 ± 0.005 %	1.286 ± 0.016 %
7.50	99.96%	1.003666 ± 0.000345	0.080484 ± 0.007667	0.002463 ± 0.000017	0.393500	0.044364	-0.016 ± 0.005 %	1.201 ± 0.018 %
7.75	99.97%	1.002205 ± 0.000312	0.048408 ± 0.006873	0.002002 ± 0.000013	0.363822	0.035939	-0.038 ± 0.004 %	1.070 ± 0.017 %
7.80	99.97%	1.001824 ± 0.000313	0.040051 ± 0.006892	0.001974 ± 0.000012	0.194758	0.035334	-0.030 ± 0.004 %	1.075 ± 0.014 %
8.00	99.97%	1.001736 ± 0.000304	0.038111 ± 0.006704	0.001844 ± 0.000014	0.562781	0.031040	-0.030 ± 0.004 %	1.031 ± 0.023 %
8.25	99.97%	1.001911 ± 0.000288	0.041960 ± 0.006350	0.001549 ± 0.000012	0.438731	0.028080	-0.017 ± 0.004 %	0.948 ± 0.020 %
8.30	99.98%	1.001561 ± 0.000280	0.034274 ± 0.006185	0.001485 ± 0.000010	0.330964	0.026834	-0.011 ± 0.004 %	0.923 ± 0.017 %
8.50	99.98%	1.001301 ± 0.000265	0.028563 ± 0.005830	0.001258 ± 0.000018	0.926594	0.022297	-0.016 ± 0.004 %	0.885 ± 0.036 %
8.75	99.98%	1.002038 ± 0.000232	0.044738 ± 0.005156	0.000886 ± 0.000006	0.168904	0.012202	-0.020 ± 0.003 %	0.713 ± 0.012 %
8.80	99.98%	1.002025 ± 0.000232	0.044443 ± 0.005160	0.000881 ± 0.000006	0.183456	0.011982	-0.021 ± 0.003 %	0.715 ± 0.013 %
9.00	99.98%	1.001946 ± 0.000227	0.042725 ± 0.005036	0.000830 ± 0.000004	0.080291	0.010794	-0.013 ± 0.003 %	0.672 ± 0.007 %
9.25	99.98%	1.001806 ± 0.000225	0.039656 ± 0.004985	0.000805 ± 0.000004	0.133142	0.010662	-0.012 ± 0.003 %	0.669 ± 0.009 %
9.30	99.98%	1.001752 ± 0.000225	0.038468 ± 0.004984	0.000792 ± 0.000004	0.100964	0.010653	-0.012 ± 0.003 %	0.676 ± 0.009 %
9.50	99.98%	1.001729 ± 0.000221	0.037948 ± 0.004912	0.000770 ± 0.000004	0.070395	0.009889	-0.010 ± 0.003 %	0.659 ± 0.008 %
9.75	99.99%	1.001773 ± 0.000218	0.038927 ± 0.004847	0.000745 ± 0.000004	0.070781	0.010044	-0.012 ± 0.003 %	0.649 ± 0.008 %
9.80	99.99%	1.001839 ± 0.000217	0.040373 ± 0.004831	0.000741 ± 0.000004	0.144547	0.008833	-0.013 ± 0.003 %	0.652 ± 0.010 %
10.00	99.99%	1.001912 ± 0.000214	0.041973 ± 0.004767	0.000704 ± 0.000004	0.142314	0.008692	-0.011 ± 0.003 %	0.637 ± 0.010 %
10.25	99.99%	1.001895 ± 0.000212	0.041595 ± 0.004703	0.000671 ± 0.000005	0.192968	0.009046	-0.010 ± 0.003 %	0.631 ± 0.012 %
10.30	99.99%	1.001793 ± 0.000211	0.039367 ± 0.004686	0.000663 ± 0.000004	0.080132	0.008241	-0.011 ± 0.002 %	0.614 ± 0.007 %
10.50	99.99%	1.001711 ± 0.000208	0.037563 ± 0.004625	0.000647 ± 0.000003	0.047110	0.008988	-0.009 ± 0.002 %	0.611 ± 0.007 %
10.75	99.99%	1.001667 ± 0.000204	0.036600 ± 0.004524	0.000602 ± 0.000003	0.055274	0.008006	-0.011 ± 0.002 %	0.585 ± 0.007 %
10.80	99.99%	1.001796 ± 0.000204	0.039420 ± 0.004538	0.000601 ± 0.000003	0.043833	0.007718	-0.017 ± 0.002 %	0.580 ± 0.006 %
11.00	99.99%	1.001853 ± 0.000202	0.040683 ± 0.004499	0.000589 ± 0.000003	0.076128	0.007433	-0.014 ± 0.002 %	0.572 ± 0.006 %
11.25	99.99%	1.001751 ± 0.000201	0.038444 ± 0.004463	0.000568 ± 0.000003	0.052181	0.007172	-0.011 ± 0.002 %	0.563 ± 0.005 %
11.30	99.99%	1.001664 ± 0.000201	0.036527 ± 0.004456	0.000563 ± 0.000003	0.078151	0.006822	-0.008 ± 0.002 %	0.566 ± 0.006 %
11.50	99.99%	1.001737 ± 0.000199	0.038121 ± 0.004414	0.000550 ± 0.000003	0.057737	0.006762	-0.010 ± 0.002 %	0.552 ± 0.006 %
11.75	99.99%	1.001612 ± 0.000198	0.035386 ± 0.004396	0.000536 ± 0.000003	0.061081	0.006804	-0.009 ± 0.002 %	0.547 ± 0.005 %
11.80	99.99%	1.001622 ± 0.000196	0.035598 ± 0.004363	0.000530 ± 0.000003	0.063305	0.006763	-0.008 ± 0.002 %	0.542 ± 0.006 %