IsValorum/Clover-35B-A3B-Preview-Q8_0-GGUF

Clover-35B-A3B-Preview Q8_0 GGUF

Clover quantization by IsValorum

Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.

If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.

Q8_0 of Diffuspace/Clover-35B-A3B-Preview at revision 4b0402b65e523d0183a14c431955438714cb16ff, from IsValorum/Clover-35B-A3B-Preview-BF16-GGUF.

This file does not load in vanilla llama.cpp. Clover uses the clover_moe architecture, which upstream does not implement. Use the precompiled builds in IsValorum/llama.cpp-clover-moe, which ships Linux x86_64 CPU and CUDA 12 builds plus a Windows x86_64 universal package (NVIDIA CUDA 12.8, Vulkan for AMD and Intel GPUs, and CPU), or compile the runtime from the patched branch (PR #31306, commit 4838ffb0c8ade92f34d9157deea38d92758ab824). The CUDA builds need an NVIDIA GPU; the CPU builds run on any x86-64 machine.

The Linux x86_64 archives require GLIBC_2.38 or newer, so run them on Ubuntu 24.04 or equivalent; the Windows package has no glibc requirement.

What the prebuilt runtimes cover. Linux x86_64: any CPU, and NVIDIA CUDA 12 for sm_80/86/89/90. Windows x86_64, in a single universal package: NVIDIA CUDA 12.8 from sm_50 through sm_120, AMD and Intel GPUs through Vulkan 1.4.363, and any CPU with per-CPU backends. Not covered: macOS, ARM64, ROCm, and Blackwell on Linux; those need a build from the same branch and commit.

File	Size	SHA256
Clover-35B-A3B-Preview-Q8_0.gguf	34.37 GiB	3d2ded3e5604c0e05c2956f95bd864a3bb6635c1442f23318484877e22cd52ed

export LD_LIBRARY_PATH=/path/to/llama-clover-bin:$LD_LIBRARY_PATH
llama-cli -m Clover-35B-A3B-Preview-Q8_0.gguf -ngl 99 -fa on -c 4096 -p "Hello"

Drop -ngl 99 on the CPU build.

A 1.25M I-Matrix is published separately.