Instructions to use Diffuspace/Clover-35B-A3B-Preview-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Diffuspace/Clover-35B-A3B-Preview-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Diffuspace/Clover-35B-A3B-Preview-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Ollama
How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with Ollama:
```
ollama run hf.co/Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with Docker Model Runner:
```
docker model run hf.co/Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M
```

Lemonade

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Clover-35B-A3B-Preview-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Diffuspace/Clover-35B-A3B-Preview-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Clover-35B-A3B-Preview-GGUF / README.md

Sansa

Refine GGUF model card

32e22a0 verified 12 days ago

preview code

Raw History Blame Contribute Delete

7.77 kB

---
license: apache-2.0
base_model: Diffuspace/Clover-35B-A3B-Preview
base_model_relation: quantized
library_name: llama.cpp
pipeline_tag: text-generation
tags:
  - gguf
  - clover
  - diffusion
  - mixture-of-experts
  - self-speculative-decoding
---
<p align="center">
  <a href="https://diffuspace.ai/">
    <img src="https://assets.diffuspace.ai/brand/logofull.svg" alt="DiffuSpace" width="360" />
  </a>
</p>
# Clover-35B-A3B-Preview-GGUF

**Clover-35B-A3B-Preview** is the first model released by **DiffuSpace**, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our [launch blog](https://diffuspace.ai/blog/clover-35b/). Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a **128K context window**.

This repository provides the **official GGUF quantizations** for local inference with our [llama.cpp integration](https://github.com/ggml-org/llama.cpp/pull/31306). The 8-bit and 4-bit variants support **self-speculative decoding** using a single model, without separate draft-model weights.

[Blog](https://diffuspace.ai/blog/clover-35b/) · [Code](https://github.com/DiffuSpace/Clover-35B-A3B-Preview) · [BF16 model](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview) · [llama.cpp PR](https://github.com/ggml-org/llama.cpp/pull/31306)

## Performance

[![Clover agentic benchmark results](https://raw.githubusercontent.com/DiffuSpace/Clover-35B-A3B-Preview/main/benchmark.png)](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks)

See the [launch blog](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for evaluation settings and comparisons. These results describe the original model; they are not separate evaluations of the GGUF quantizations. Quantization and inference settings can affect quality and speed.

## Available quantizations

Download only the variant you want to run; each file is a complete model.

| File | Size | Description |
| --- | ---: | --- |
| [Clover-35B-A3B-Preview-Q8_0.gguf](./Clover-35B-A3B-Preview-Q8_0.gguf) | 35.26 GiB | Primarily 8-bit quantization; the largest of the three variants. |
| [Clover-35B-A3B-Preview-Q4_K_M.gguf](./Clover-35B-A3B-Preview-Q4_K_M.gguf) | 21.42 GiB | Mixed quantization using Q4_K, Q6_K, Q8_0, and floating-point tensors. |
| [Clover-35B-A3B-Preview-Q4_0.gguf](./Clover-35B-A3B-Preview-Q4_0.gguf) | 18.24 GiB | Q4_0 quantization with some tensors retained in floating point; the smallest download. |

Sizes above use GiB (2³⁰ bytes); the corresponding decimal sizes are approximately 37.86 GB, 23.00 GB, and 19.59 GB. File checksums are provided in [SHA256SUMS](./SHA256SUMS).

## Architecture and decoding

Clover uses a hybrid Gated DeltaNet (GDN) / full-attention MoE backbone. Additional learned GDN parameters support bidirectional attention within diffusion blocks while preserving a causal prefix.

The llama.cpp integration provides **self-speculative decoding** through `draft-self`: Clover proposes tokens with its block diffusion capability and verifies them with its own causal decoding path. A separate draft model is not required.

The supported context window is **131,072 tokens**, including both the prompt and generated response. The GGUF metadata records this limit. Start with a smaller runtime context and increase it as needed to fit your workload and memory budget.

## Local and on-device inference

GGUF makes Clover available for local deployment through the linked llama.cpp integration. Choose a quantization according to the memory available on your machine. Q8_0 uses more memory to retain higher precision; Q4_K_M provides a mixed-precision option; Q4_0 has the smallest weight footprint.

**File size is not total inference memory.** Allow additional RAM or VRAM for attention caches, recurrent state, and compute buffers. Longer contexts increase memory requirements. Approximately 3B parameters are active per token, but the model still contains approximately 35B parameters of weights.

The example below uses NVIDIA CUDA. A CPU-only build is also shown. Actual speed depends on the hardware, quantization, prompt length, and number of accepted draft tokens.

## Quick start

Use the Clover implementation in [llama.cpp PR #31306](https://github.com/ggml-org/llama.cpp/pull/31306). The commands below fetch that PR explicitly so the build includes Clover support.

### 1. Get the source

```bash
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
git fetch origin pull/31306/head
git checkout --detach FETCH_HEAD
```

### 2. Build

For an NVIDIA GPU, install a compatible CUDA toolkit and build the speculative decoding example:

```bash
cmake -S . -B build-clover \
  -DGGML_CUDA=ON \
  -DLLAMA_CURL=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-clover --config Release --target llama-speculative-simple -j
```

<details>
<summary>CPU-only build</summary>

```bash
cmake -S . -B build-clover-cpu \
  -DGGML_CUDA=OFF \
  -DGGML_BLAS=OFF \
  -DLLAMA_CURL=OFF \
  -DLLAMA_BUILD_TESTS=OFF \
  -DLLAMA_BUILD_SERVER=OFF \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-clover-cpu --config Release --target llama-speculative-simple -j
```

Use `./build-clover-cpu/bin/llama-speculative-simple` in the run command below and set `-ngl 0`.

</details>

### 3. Download a model

From the llama.cpp directory, download one GGUF file:

```bash
pip install -U huggingface_hub
hf download Diffuspace/Clover-35B-A3B-Preview-GGUF \
  Clover-35B-A3B-Preview-Q4_K_M.gguf \
  --local-dir models
```

Replace the filename with the Q8_0 or Q4_0 variant to use a different quantization.

### 4. Run self-speculative decoding

The `llama-speculative-simple` example reads a raw prompt, so include the model's chat formatting explicitly:

```bash
cat > prompt.txt <<'EOF'
<|im_start|>user
What is diffusion?<|im_end|>
<|im_start|>assistant
<think>
EOF
./build-clover/bin/llama-speculative-simple \
  -m models/Clover-35B-A3B-Preview-Q4_K_M.gguf \
  -f prompt.txt \
  --spec-type draft-self \
  --spec-draft-n-max 15 \
  --spec-accept-ratio 0.9 \
  --spec-accept-delta 0.6 \
  --temp 0 --top-k 1 --seed 42 \
  -ctk bf16 -ctv bf16 \
  -c 8192 -n 4096 -b 256 -ub 256 \
  -ngl 99 -t 8 -tb 8 \
  --flash-attn on --color off
```

This example uses greedy generation and the PR's entropy-based draft acceptance settings. Keep those acceptance options with the `llama-speculative-simple` executable; they are not general-purpose flags for every llama.cpp frontend.

`-c` sets the runtime context size, and `-n` limits newly generated tokens, including the thinking phase. Increase them for longer tasks while keeping the prompt and response within the **128K** limit. This example executable also requires the prompt to fit within `-b`; increase the batch size for longer prompts. `-ngl 99` requests GPU offloading for all model layers, so ensure sufficient VRAM for the selected model and context.

For the standard Hugging Face inference path, see the [BF16 model card](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview). For serving and evaluation with SGLang, see the [official code repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

## License and attribution

Clover is derived from Qwen3.6-35B-A3B and released under **Apache-2.0**. The accompanying [LICENSE](./LICENSE) preserves the upstream license and copyright statement. These GGUF files are quantized versions of the official Clover model; see [ATTRIBUTIONS.md](./ATTRIBUTIONS.md) for provenance and modification notices.

[llama.cpp](https://github.com/ggml-org/llama.cpp) is distributed separately under its own MIT license.

	---
	license: apache-2.0
	base_model: Diffuspace/Clover-35B-A3B-Preview
	base_model_relation: quantized
	library_name: llama.cpp
	pipeline_tag: text-generation
	tags:
	- gguf
	- clover
	- diffusion
	- mixture-of-experts
	- self-speculative-decoding
	---

	<p align="center">
	<a href="https://diffuspace.ai/">
	<img src="https://assets.diffuspace.ai/brand/logofull.svg" alt="DiffuSpace" width="360" />
	</a>
	</p>

	# Clover-35B-A3B-Preview-GGUF

	Clover-35B-A3B-Preview is the first model released by DiffuSpace, delivering state-of-the-art performance among open-source diffusion language models at release on the benchmarks reported in our [launch blog](https://diffuspace.ai/blog/clover-35b/). Clover combines a 35B-parameter mixture-of-experts backbone with approximately 3B active parameters per token, block diffusion generation, and a 128K context window.

	This repository provides the official GGUF quantizations for local inference with our [llama.cpp integration](https://github.com/ggml-org/llama.cpp/pull/31306). The 8-bit and 4-bit variants support self-speculative decoding using a single model, without separate draft-model weights.

	[Blog](https://diffuspace.ai/blog/clover-35b/) · [Code](https://github.com/DiffuSpace/Clover-35B-A3B-Preview) · [BF16 model](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview) · [llama.cpp PR](https://github.com/ggml-org/llama.cpp/pull/31306)

	## Performance

	[![Clover agentic benchmark results](https://raw.githubusercontent.com/DiffuSpace/Clover-35B-A3B-Preview/main/benchmark.png)](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks)

	See the [launch blog](https://diffuspace.ai/blog/clover-35b/#agentic-benchmarks) for evaluation settings and comparisons. These results describe the original model; they are not separate evaluations of the GGUF quantizations. Quantization and inference settings can affect quality and speed.

	## Available quantizations

	Download only the variant you want to run; each file is a complete model.

	\| File \| Size \| Description \|
	\| --- \| ---: \| --- \|
	\| [Clover-35B-A3B-Preview-Q8_0.gguf](./Clover-35B-A3B-Preview-Q8_0.gguf) \| 35.26 GiB \| Primarily 8-bit quantization; the largest of the three variants. \|
	\| [Clover-35B-A3B-Preview-Q4_K_M.gguf](./Clover-35B-A3B-Preview-Q4_K_M.gguf) \| 21.42 GiB \| Mixed quantization using Q4_K, Q6_K, Q8_0, and floating-point tensors. \|
	\| [Clover-35B-A3B-Preview-Q4_0.gguf](./Clover-35B-A3B-Preview-Q4_0.gguf) \| 18.24 GiB \| Q4_0 quantization with some tensors retained in floating point; the smallest download. \|

	Sizes above use GiB (2³⁰ bytes); the corresponding decimal sizes are approximately 37.86 GB, 23.00 GB, and 19.59 GB. File checksums are provided in [SHA256SUMS](./SHA256SUMS).

	## Architecture and decoding

	Clover uses a hybrid Gated DeltaNet (GDN) / full-attention MoE backbone. Additional learned GDN parameters support bidirectional attention within diffusion blocks while preserving a causal prefix.

	The llama.cpp integration provides self-speculative decoding through `draft-self`: Clover proposes tokens with its block diffusion capability and verifies them with its own causal decoding path. A separate draft model is not required.

	The supported context window is 131,072 tokens, including both the prompt and generated response. The GGUF metadata records this limit. Start with a smaller runtime context and increase it as needed to fit your workload and memory budget.

	## Local and on-device inference

	GGUF makes Clover available for local deployment through the linked llama.cpp integration. Choose a quantization according to the memory available on your machine. Q8_0 uses more memory to retain higher precision; Q4_K_M provides a mixed-precision option; Q4_0 has the smallest weight footprint.

	File size is not total inference memory. Allow additional RAM or VRAM for attention caches, recurrent state, and compute buffers. Longer contexts increase memory requirements. Approximately 3B parameters are active per token, but the model still contains approximately 35B parameters of weights.

	The example below uses NVIDIA CUDA. A CPU-only build is also shown. Actual speed depends on the hardware, quantization, prompt length, and number of accepted draft tokens.

	## Quick start

	Use the Clover implementation in [llama.cpp PR #31306](https://github.com/ggml-org/llama.cpp/pull/31306). The commands below fetch that PR explicitly so the build includes Clover support.

	### 1. Get the source

	```bash
	git clone https://github.com/ggml-org/llama.cpp.git
	cd llama.cpp
	git fetch origin pull/31306/head
	git checkout --detach FETCH_HEAD
	```

	### 2. Build

	For an NVIDIA GPU, install a compatible CUDA toolkit and build the speculative decoding example:

	```bash
	cmake -S . -B build-clover \
	-DGGML_CUDA=ON \
	-DLLAMA_CURL=OFF \
	-DLLAMA_BUILD_TESTS=OFF \
	-DLLAMA_BUILD_SERVER=OFF \
	-DCMAKE_BUILD_TYPE=Release
	cmake --build build-clover --config Release --target llama-speculative-simple -j
	```

	<details>
	<summary>CPU-only build</summary>

	```bash
	cmake -S . -B build-clover-cpu \
	-DGGML_CUDA=OFF \
	-DGGML_BLAS=OFF \
	-DLLAMA_CURL=OFF \
	-DLLAMA_BUILD_TESTS=OFF \
	-DLLAMA_BUILD_SERVER=OFF \
	-DCMAKE_BUILD_TYPE=Release
	cmake --build build-clover-cpu --config Release --target llama-speculative-simple -j
	```

	Use `./build-clover-cpu/bin/llama-speculative-simple` in the run command below and set `-ngl 0`.

	</details>

	### 3. Download a model

	From the llama.cpp directory, download one GGUF file:

	```bash
	pip install -U huggingface_hub
	hf download Diffuspace/Clover-35B-A3B-Preview-GGUF \
	Clover-35B-A3B-Preview-Q4_K_M.gguf \
	--local-dir models
	```

	Replace the filename with the Q8_0 or Q4_0 variant to use a different quantization.

	### 4. Run self-speculative decoding

	The `llama-speculative-simple` example reads a raw prompt, so include the model's chat formatting explicitly:

	```bash
	cat > prompt.txt <<'EOF'
	<\|im_start\|>user
	What is diffusion?<\|im_end\|>
	<\|im_start\|>assistant
	<think>
	EOF

	./build-clover/bin/llama-speculative-simple \
	-m models/Clover-35B-A3B-Preview-Q4_K_M.gguf \
	-f prompt.txt \
	--spec-type draft-self \
	--spec-draft-n-max 15 \
	--spec-accept-ratio 0.9 \
	--spec-accept-delta 0.6 \
	--temp 0 --top-k 1 --seed 42 \
	-ctk bf16 -ctv bf16 \
	-c 8192 -n 4096 -b 256 -ub 256 \
	-ngl 99 -t 8 -tb 8 \
	--flash-attn on --color off
	```

	This example uses greedy generation and the PR's entropy-based draft acceptance settings. Keep those acceptance options with the `llama-speculative-simple` executable; they are not general-purpose flags for every llama.cpp frontend.

	`-c` sets the runtime context size, and `-n` limits newly generated tokens, including the thinking phase. Increase them for longer tasks while keeping the prompt and response within the 128K limit. This example executable also requires the prompt to fit within `-b`; increase the batch size for longer prompts. `-ngl 99` requests GPU offloading for all model layers, so ensure sufficient VRAM for the selected model and context.

	For the standard Hugging Face inference path, see the [BF16 model card](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview). For serving and evaluation with SGLang, see the [official code repository](https://github.com/DiffuSpace/Clover-35B-A3B-Preview).

	## License and attribution

	Clover is derived from Qwen3.6-35B-A3B and released under Apache-2.0. The accompanying [LICENSE](./LICENSE) preserves the upstream license and copyright statement. These GGUF files are quantized versions of the official Clover model; see [ATTRIBUTIONS.md](./ATTRIBUTIONS.md) for provenance and modification notices.

	[llama.cpp](https://github.com/ggml-org/llama.cpp) is distributed separately under its own MIT license.