---
base_model: bottlecapai/ThinkingCap-Qwen3.8-Flash-Next
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
tags:
  - auto-round
  - int4
  - fp8
  - mixed-precision
  - qwen4_exp
---

# ThinkingCap-Qwen3.8-Flash-Next-W4A16-AutoRound-FP8NGram

Weights from [bottlecapai/ThinkingCap-Qwen3.8-Flash-Next](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next),
quantized using the recipe from
[Intel/Qwen3.8-Flash-Next-W4A16-AutoRound](https://huggingface.co/Intel/Qwen3.8-Flash-Next-W4A16-AutoRound).
Intel supplies the recipe, **not replacement weights**. ThinkingCap's tokenizer,
chat template and vision processor are retained.

**Only intended precision deviation from Intel: ngram lookup weights use
scaled FP8 E4M3FN instead of BF16.** All other precision settings and exclusions
are unchanged.

## Layer precision

| Component | Checkpoint precision / treatment |
| --- | --- |
| Main routed MoE experts: `gate_proj`, `up_proj`, `down_proj` | Symmetric INT4, group size 128; W4A16 |
| Ngram lookup tables | FP8 E4M3FN; shared FP32 scale per logical table across splits |
| Other PLE weights and projections | Original 16-bit precision; BF16 source |
| Self/linear attention, `in_proj_a`, `in_proj_b`, indexers | Original 16-bit precision; BF16 source |
| Routers (`mlp.gate`), shared experts and their gates | Original 16-bit precision; BF16 source |
| Hyper-connection weights | Original 16-bit precision; BF16 source |
| Vision encoder and merger | Original precision; excluded |
| Token embeddings and LM head | Original precision; excluded |
| Entire MTP module, including experts and attention | Original precision; excluded |
| Norms, hash/index buffers, other non-target tensors | Original dtype; integer buffers remain integer |

W4A16 means 4-bit weights and 16-bit activations for INT4 targets. No FP8
activation conversion or FP8 KV-cache requirement is introduced; runtime
compute/cache dtypes depend on the inference engine.

## Quantization recipe

Uses [Intel AutoRound](https://github.com/intel/auto-round), integrated into
[Intel Neural Compressor](https://github.com/intel/neural-compressor).

| Setting | Value |
| --- | --- |
| INT4 recipe | W4A16, symmetric, group size 128, 200 tuning iterations |
| Calibration | Batch 8, sequence length 2048, 128 samples, seed 42; default dataset |
| Device / export | `auto` / `auto_round` (`auto_round:auto_gptq` packing) |
| Implementation | AutoRound 0.16.1 for compatibility fixes; Intel used 0.15.0 |

Same configured INT4 recipe, not a claim of bit-identical weights. Intel's
exclusion list is retained:

```text
lm_head,embed_tokens,visual,linear_attn,self_attn,hyper_connection,mlp.gate,shared_expert,in_proj_a,in_proj_b,ple,mtp,indexer
```

After INT4 export, `*.ngram_embedding[.shard_N].weight` is converted using
`FP8(clamp(weight / scale, -448, 448))`, where `scale = amax / 448` (1 for zero
tables). Splits share an FP32 `*.ngram_embedding.weight_scale`;
`text_config.ple_embedding_dtype` is set to `float8_e4m3fn`.

## Provenance

- ThinkingCap source revision: `d90c9fedadaf7556df4206e6bea5a6b391b1a58e`.
- Intel recipe revision: `4c67bf686b7f7fd386bae6b07ab59e8ff1d5b897`.
- Export/conversion details and package versions: `quantization-run.json`.
- Attribution does not imply endorsement by Intel, bottlecapai or Qwen.

## Inference and limitations

Requires AutoRound INT4 and FP8 PLE support, including shared scales and split
tables; vLLM 0.32.0 implements these loading paths. Compatibility is not universal.
No benchmarks are claimed; Intel's scores do not describe this ThinkingCap
variant. Validate quality, multimodal behavior and serving compatibility.
Source-model limitations and safety considerations still apply.

## License

**Qwen Community License 1.0**, unchanged from
[ThinkingCap](https://huggingface.co/bottlecapai/ThinkingCap-Qwen3.8-Flash-Next/blob/d90c9fedadaf7556df4206e6bea5a6b391b1a58e/LICENSE);
the original [LICENSE](LICENSE) is retained, not replaced by the quantizer's
software license. Commercial conditions include separate licensing for certain
Model as a Service and AI Work Assistant businesses. Read the full license;
this is not legal advice.