PocketAiHub/Nanbeige4.3-33B-MLX-6bit

Nanbeige 4.3 33B — PocketAiHub MLX 6-bit

An independently converted, faithful quantization of Nanbeige/Nanbeige4.3-33B. The original model is by the Nanbeige Team. PocketAiHub provides conversion, validation and distribution; this is not an official Nanbeige release. No fine-tuning, abliteration or other behavioral editing was applied.

• Original revision: c1093c501cc6460427c0ac634db7223d0450c0ee.
• Recipe: uniform 6-bit for quantizable modules; affine quantization, group size 64.
• Candidate payload: 26.929 GB (decimal; runtime memory is additional).
• MLX 0.32.2; MLX-LM 0.31.3.
• Chat template, tokenizer, reasoning controls and RoPE settings retained.
• Original license: Apache 2.0; see LICENSE.

Use

mlx_lm.generate --model PocketAiHub/Nanbeige4.3-33B-MLX-6bit --prompt "Write a Python retry decorator."

Upstream recommends temperature 0.6/top-p 0.95 for reasoning/chat and temperature 1.0/top-p 0.95 for agents/tool use. Thinking is enabled by default. For multi-turn agents preserve previous thinking and use XML tool-call format. Compatible runtimes must support the original chat-template options.

Measured local regression

Hardware: Apple M5 Max, 128 GiB unified memory. The local suite uses greedy decoding with thinking disabled except its explicit thinking probe. It covers 80 capability cases, 16 XML/JSON tool routes, two Chinese cases, a thinking probe and an approximately 8K-token retrieval probe. Exact prompts, output, token counts and failures are included under evaluation/.

Check	This quant	BF16 reference
Local regression cases passed	90/100	88/100
Truncated outputs	1	1

Known inherited limitation: in greedy non-thinking mode, the BF16 reference loops on the suite's short repeated-syllable instruction until the token cap. Any occurrence in this quant is counted above and preserved in the raw records. The conversion gate permits inherited truncations and rejects new ones; it does not claim to repair the source model. Use the upstream default thinking mode and suitable generation limits for deployment.

Held-out WikiText drift: mean KL 0.033143, top-1 agreement 92.794%, measured against MLX BF16 on eight 512-token chunks. This is a small conversion regression, not upstream benchmark reproduction or proof of general quality parity. See qualification.json for the exact thresholds and new failures relative to BF16.

The upstream architecture supports 512K context. Full 512K operation was not validated here; the context probe tests approximately 8K prompt tokens on the stated hardware. Longer contexts require additional KV-cache memory.

Reproducibility

artifact-manifest.json records source lineage, conversion command, versions, module recipe and SHA-256 hashes. release-manifest.json covers the staged payload and evidence. Raw calibration and WikiText corpora are not redistributed.