Gemma 4 31B Larkspur v0.5 — Q4_K_M GGUF
Q4_K_M quantization of trashpanda-org/gemma-4-31b-larkspur-v0.5.
Quant Details
| Property | Value |
|---|---|
| Source | trashpanda-org/gemma-4-31b-larkspur-v0.5 |
| Quant | Q4_K_M (4.83 BPW) |
| Size | ~18 GB |
| Format | GGUF (llama.cpp) |
| Original Precision | bf16 |
Usage
Load with any llama.cpp-compatible runtime (llama.cpp, KoboldCpp, ollama, LM Studio, etc.):
llama-cli -m gemma-4-31b-larkspur-v0.5-Q4_K_M.gguf -p "Your prompt here"
Notes
- Quantized from the bf16 source weights using llama.cpp's
convert_hf_to_gguf.py→llama-quantize - Q4_K_M offers a good balance of quality and size, fitting comfortably on 2×T4 (32 GB) with room for KV cache
- Downloads last month
- 27
Hardware compatibility
Log In to add your hardware
4-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for finis-est/gemma-4-31b-larkspur-v0.5-Q4_K_M
Base model
trashpanda-org/gemma-4-31b-larkspur-v0.5