smalinin/MiniMax-M3.1-Flash-Q2_K-GGUF

MiniMax-M3.1-Flash Q2_K GGUF

A mixed-precision GGUF quantization of MiniMax-M3.1-Flash, with a separate vision projector and DSpark speculative decoding model.

The main model uses the Q2_K-Protected recipe: routed expert gate/up matrices use Q2_K, while expert down matrices and the attention/dense/shared paths receive higher precision. The uploaded filenames use the shorter Q2_K name; this is a custom mixed recipe, rather than the default uniform Q2_K preset.

The original model accepts text, images and video and uses MiniMax Sparse Attention (MSA). See the original model card for its architecture, intended uses and upstream evaluation results. Those scores do not measure this quantization.

Files and download size

Component	Location	Size
Main Q2_K-Protected model	Q2_K/, all six shards	162.266 GB
Vision projector, BF16/F32	mmproj-MiniMax-M3.1-Flash-BF16.gguf	1.735 GB
DSpark draft model, BF16/F32	MiniMax-M3.1-Flash-DSpark-BF16.gguf	3.925 GB
Alternative DFlash2 draft model, BF16	MiniMax-M3.1-Flash-DFlash2-BF16.gguf	4.257 GB

The main model, vision projector and DSpark together occupy 156.394 GiB on disk. DFlash2 is an alternative draft model and is not needed for DSpark.

Download all six main shards into the same directory. Pass only the first shard to llama.cpp; it finds the remaining files automatically:

Q2_K/MiniMax-M3.1-Flash-Q2_K-00001-of-00006.gguf
...
Q2_K/MiniMax-M3.1-Flash-Q2_K-00006-of-00006.gguf

Quantization recipe

Tensor group	Storage type	Tensor count
Routed expert gate/up	Q2_K	114
Routed expert down	Q3_K	57
Attention Q/K/V/output, dense/shared FFN down, output head	Q6_K	301
Dense/shared FFN gate/up	Q5_K	120
Token embedding	BF16	1
MSA indexer, routers, norms and auxiliary scales	F32	766
Total		1359

Conversion starts from a GGUF that preserves the original NVFP4 expert codes/scales and stores the other main matrices in BF16. Expert matrices are dequantized from NVFP4 and requantized directly into their final Q2_K/Q3_K types. There is no intermediate MXFP4 or IQ3_S quantization in this path. Requantization introduces additional error relative to the released checkpoint.

Calibration uses the MiniMax-M3.1-Flash importance matrix from Unsloth: 768 entries, metadata reporting 250 chunks of 8192 tokens, with coverage of all 57 MoE layers and 128 experts per layer. The imatrix is used during quantization; it is not needed for inference.

This is an experimental low-bit recipe. No held-out perplexity or task benchmark scores are published for this quantization. Tensor-layout checks and conversion checks do not establish equivalence to the original model.

Runtime

Use the MiniMax-enabled smalinin/llama.cpp my_build branch. The implementation includes MSA, quantized KV cache with Flash Attention, the vision projector and DSpark integration.

Reference revision: 0a56bfd856ce51939e6c5d731df2a99dd6d5d2d0, including recovery after failed graph-buffer allocation. This revision includes the preceding MiniMax and DSpark cache-reuse changes.

Example CUDA build, with CMake, a C/C++ compiler and the CUDA toolkit installed:

git clone --branch my_build https://github.com/smalinin/llama.cpp.git
cd llama.cpp
cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --target llama-server -j

Download

Using the Hugging Face CLI:

hf download smalinin/MiniMax-M3.1-Flash-Q2_K-GGUF \
  --include 'Q2_K/*.gguf' \
            'mmproj-MiniMax-M3.1-Flash-BF16.gguf' \
            'MiniMax-M3.1-Flash-DSpark-BF16.gguf' \
  --local-dir ./MiniMax-M3.1-Flash-Q2_K-GGUF

Serve with vision and DSpark

Set the two paths below to the server binary and downloaded model directory. This example starts with an 8192-token context and one slot; adapt GPU placement to your hardware.

LLAMA_SERVER=/path/to/llama.cpp/build/bin/llama-server
MODEL_DIR=/path/to/MiniMax-M3.1-Flash-Q2_K-GGUF

"$LLAMA_SERVER" \
  -m "$MODEL_DIR/Q2_K/MiniMax-M3.1-Flash-Q2_K-00001-of-00006.gguf" \
  --mmproj "$MODEL_DIR/mmproj-MiniMax-M3.1-Flash-BF16.gguf" \
  -md "$MODEL_DIR/MiniMax-M3.1-Flash-DSpark-BF16.gguf" \
  --spec-type draft-dspark \
  --gpu-layers-draft all \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0 \
  --n-cpu-moe 0 \  
  -c 8192 -np 1 \
  -b 256 -ub 64 -t 12 \
  -fa on -ctk q8_0 -ctv q8_0 \
  --temp 1.0 --top-p 0.95 \
  --load-mode mmap --lazy-mode on \
  --jinja --no-warmup \
  --chat-template-kwargs '{"reasoning_effort":"high","enable_thinking":true}' \
  --alias minimax-m31 \
  --host 127.0.0.1 --port 8085

The projector is optional for text-only inference. Omit --mmproj when images are not needed. For inference without speculative decoding, omit -md and all --spec-*/--gpu-layers-draft options. DSpark is a separate model; it is not embedded in the main six shards. Its benefit depends on prompt, acceptance rate, context length and hardware; no speedup is claimed for this Q2_K quantization.

The projector was checked for preservation of decoded source tensor values. Image QA, OCR and short video tests with DSpark were exercised in this runtime using the separate MXFP4/Q8 target model; these results are not a quality evaluation of the Q2_K target. Video input additionally requires the runtime's FFmpeg integration. Native MiniMax video timestamp formatting is not implemented in this revision.

DFlash2 is included as an experimental alternative. Its GGUF conversion was checked against the source checkpoint, but end-to-end generation with it has not been validated here. The command above uses DSpark.

KV cache and memory requirements

Model-file size is only one part of memory use. GPU memory also holds the KV cache, graph/compute buffers, backend overhead and any offloaded vision/draft models. CPU offloading changes placement and performance.

For the main model, this MSA implementation stores history for all 60 layers. Each layer has four KV heads of dimension 128, plus a separate MSA indexer key cache with one head of dimension 128. MSA selects sparse attention blocks but does not discard the full stored K/V history in this runtime. The indexer has no value cache.

For one sequence, the main model's cache payload per token is:

Main K/V: 60 layers * 4 heads * 128 dimensions * 2 (K and V)
Indexer K: 60 layers * 1 head * 128 dimensions
Total: 69,120 stored scalar values per token

FP16:  69,120 * 2 bytes       = 138,240 bytes/token
Q8_0:  69,120 * 34/32 bytes  =  73,440 bytes/token

Q8_0 stores 32 quantized values and a two-byte scale in each 34-byte block. Its scale overhead is included in these numbers.

Example: 128k context

Here 128k means -c 128000 -np 1, with the main model and indexer caches offloaded to GPU. Sizes below are theoretical tensor payloads, summed across GPUs:

Cache configuration	Main K/V	MSA indexer K	Total bytes	Total GiB
-ctk f16 -ctv f16	15,728,640,000 B	1,966,080,000 B	17,694,720,000	16.479
-ctk q8_0 -ctv q8_0	8,355,840,000 B	1,044,480,000 B	9,400,320,000	8.755

For -c 131072 (128 Ki tokens), totals are 16.875 GiB FP16 or 8.965 GiB Q8_0. At the example launch context of 8192 tokens, totals are 1.055 GiB FP16 or 0.560 GiB Q8_0.

These figures cover the main model cache, including its MSA indexer. They exclude DSpark's own cache, allocation overhead, temporary attention/indexer tensors, compute buffers and model weights. A runtime may reserve the configured cache capacity even before the context fills. Actual per-device allocation depends on placement and allocation alignment. KV storage type is independent of the Q2_K-Protected weight recipe.

For long-context prefill, keep the microbatch conservative: for example, -b 512 -ub 128, or -ub 64 when memory is tight. This reduces temporary compute-memory pressure; it does not reduce the configured KV cache capacity. The 128k figures above are cache calculations, not a claim that a complete 128k inference run fits every GPU layout. The upstream model's advertised context length is not a guarantee of usable context on a particular setup.

License and credits

Distributed under the upstream MiniMax Community License. Refer to the full license for its terms.

• Original model and DSpark: MiniMaxAI/MiniMax-M3.1-Flash.
• Calibration importance matrix: Unsloth MiniMax-M3.1-Flash-GGUF.
• Alternative DFlash2 checkpoint: incoai/MiniMax-M3.1-Flash-DFlash2.
• Quantization and MiniMax runtime integration: smalinin, using llama.cpp.

File inventory and reference runtime revision checked on 2026-10-24. GB uses 10^9 bytes; GiB uses 2^30 bytes.

Q2_S Samples

The following examples were generated with the completed files that are being uploaded.

Prompt:

create html with svg "pelican on a bicycle"

• one shoot call

Indian Railways WDP-4D dual-cab diesel locomotive using Three.js

• used opencode, toolcall, vision

create HTML page with "Indian Railways WDP-4D dual-cab diesel locomotive" using Three.js