Instructions to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
# Run inference directly in the terminal:
llama cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./llama-cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
# Run inference directly in the terminal:
./build/bin/llama-cli -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Use Docker

docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Llama new
LM Studio
Jan

vLLM

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Ollama
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with Ollama:
```
ollama run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
```
Unsloth Desktop

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with Docker Model Runner:
```
docker model run hf.co/webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS
```

Lemonade

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Run and chat with the model

lemonade run user.Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF-IQ3_XXS

List all available models

lemonade list

Hermes Agent

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF:IQ3_XXS" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Sakura Qwen3.8 Flash-Next Compact · ISTA IQ3_XXS · 512E · CP2

Experimental community repack of ISTA-DASLab GSQ-RCO IQ3_XXS, retaining all 512 routed experts. The original first GGUF shard is copied unchanged. Only the separate N-Gram/PLE lookup-table shard is replaced with the existing Sakura CP2 table.

This exact IQ3_XXS + CP2 package has not been loaded for inference or tested for generation, perplexity, quality or speed. Its pinned file hashes, GGUF split layout, lookup addressing and tokenizer metadata have been checked. Publication here is an experimental structural release, not a model-quality or hardware qualification.

Files and size

Component	Bytes	GiB
Original IQ3_XXS main weights, shard 1	47,039,860,096	43.809
Existing CP2 lookup table, shard 2	13,440,065,184	12.517
Both model files	60,479,925,280	56.326

The model files total 60.480 GB in decimal download units. The source two-shard set totals 75.840 GB; replacing its IQ4_NL table saves about 15.360 GB. The small runtime and documentation are additional. File size is not GPU memory usage or proof that this configuration fits a particular machine.

The backbone remains ISTA's original IQ3_XXS mixed quantization. CP2 describes the separate experimental table representation; this is not a stock whole-model quantization preset. Internal main-shard metadata, including the source model name and original tokenizer/template, remain unchanged.

Download and start on Windows

Download the complete repository into one folder:

hf download webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF --local-dir Sakura-IQ3_XXS-CP2

Keep both numbered GGUF files, START.cmd and the complete runtime directory together. Start START.cmd, wait for loading, then open http://127.0.0.1:18098. The OpenAI-compatible endpoint is /v1/chat/completions, with alias sakura-flash-next-iq3-xxs-cp2. Stop the server with Ctrl+C.

The proposed starter uses the included Windows x64 Vulkan CP2 runtime, one slot, context 4096, Q8 KV cache and lazy table reads from local NVMe. It preserves the source chat template and its thinking default. MTP, vision and the optional CP2 GPU row cache are disabled. This starter has been checked as a release file but has not been executed against this exact package.

Use the included runtime: stock llama.cpp, LM Studio, Ollama and unrelated forks are not qualified for this custom CP2 table. No AMD/64-GB-UMA fit, peak-memory, long-context or performance result is claimed for this variant. Settings may need adjustment for the user's actual available memory.

What was verified

The original first shard is identified by its full HF LFS SHA256 and is copied without payload or metadata changes.
The original second shard has the same full-file SHA256 as the common IQ4_NL input used by the existing Sakura CP2 repack, not merely the same tensor shape.
The original IQ4_NL table file and the existing CP2 table file were also fully rehashed locally on 2026-10-21 and matched their recorded hashes. The CP2 table is a lossy derivative, not byte-identical to IQ4_NL.
The first shard has 1,223 tensors; the replacement has one table tensor. Both use the two-shard contract for 1,224 tensors in total.
The lookup-addressing metadata and complete raw tokenizer metadata match the checked ISTA Q2 reference.
The existing CP2 runtime binaries and notices are reused unchanged from the pinned public Sakura K512 repository. No new encoder, training, expert pruning or runtime port was performed.

See results.json for scope and provenance, PROVENANCE.json for pinned sources and SHA256SUMS for all shipped files. Previously reported Sakura Q2 or Gyro-M scores are not results for this package and are not imported here. Model-file integrity after the server-side copy is checked against HF's LFS hash/size metadata; no independent full download of the new repository is performed.

Provenance and license

Base: Qwen/Qwen3.8-Flash-Next. Direct backbone: ISTA-DASLab GSQ-RCO, revision ed59f92082b1e93c0e96d60a8b11aab089b52f09. Existing CP2 shard and matching runtime: Sakura K512 CP2, revision 9b8fef370b812b3fd33787869d553793517245cb.

Credit for the GSQ-RCO quantized backbone belongs to ISTA-DASLab and its contributors; credit for Qwen belongs to Qwen. Sakura supplies the experimental lookup-table representation and this repack. This release claims no authorship of the backbone, no retraining and no endorsement by those projects.

Model weights retain Qwen Community License 1.0, included in LICENSE and LICENSE-QWEN. Read its complete conditions before use. The upstream HF card's Apache tag does not replace the inherited model-weight license here. Runtime software has its separate MIT and third-party license notices under runtime/; software licensing does not relicense model weights.

Downloads last month: 139

GGUF

Model size

177B params

Architecture

qwen4exp

Hardware compatibility

3-bit

Model tree for webmp3/Sakura-Qwen3.8-Flash-Next-Compact-ISTA-IQ3_XXS-512E-CP2-GGUF

Base model

Qwen/Qwen3.8-Flash-Next

Quantized

ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF

Quantized

(28)

this model