---
base_model: google/gemma-4-E4B-it
base_model_relation: finetune
license: apache-2.0
library_name: gguf
pipeline_tag: image-text-to-text
tags:
  - winnow
  - gemma4
  - typed-decisions
  - local-inference
  - gguf
  - decision-model
---

# Winnow-E4B

Winnow-E4B is a compact EldanRing fine-tune of [Gemma 4 E4B IT](https://huggingface.co/google/gemma-4-E4B-it) for typed decisions, chat, and image input. Give it a shared state and questions with known answer options; it scores the candidates without generating an explanation. Typical uses include routing requests, choosing actions, checking conditions, and rating urgency.

The GGUF downloads contain the merged fine-tune. No separate adapter or base-model download is needed. Image input uses the matching vision projector.

[Quickstart](https://github.com/EldanRing/winnow-inference/blob/main/docs/QUICKSTART.md) · [Inference code](https://github.com/EldanRing/winnow-inference) · [API reference](https://github.com/EldanRing/winnow-inference/blob/main/docs/API.md) · [Benchmarks](docs/BENCHMARKS.md)

## Decision results

These published Q8 results compare the models on the same questions. Jev and Kev report top-choice accuracy; Typed reports agreement with synthetic teacher labels.

| Evaluation | Winnow-E4B Q8 | Winnow-12B Q8 |
|---|---:|---:|
| JevBench public subset, 231 questions | **80.52%** (186/231) | 85.71% (198/231) |
| Kev v9 clean, 1,046 questions | **72.66%** | 81.45% |
| Kev v9 additional, 390 questions | **55.90%** | 68.97% |
| Local typed decisions, 2,000 decisions | **72.30%** | 70.10% |

Winnow-12B leads on the public Jev and Kev suites. E4B leads by 2.2 points on the local Typed teacher-agreement panel. These frozen panels were used during development. JevBench reports public-subset accuracy rather than the composite leaderboard score.

The 12B Kev-clean comparator is 852/1,046, distinct from its separate 853/1,046 release campaign. [Evaluation details](docs/BENCHMARKS.md) cover uncertainty, calibration, and scope.

![Winnow-E4B and Winnow-12B Q8 decision scores on three evaluation panels](assets/e4b-quality.png)

## Downloads

| File | Role | Size |
|---|---|---:|
| [Winnow-E4B-Q8_0.gguf](https://huggingface.co/EldanRing/Winnow-E4B/resolve/main/gguf/Winnow-E4B-Q8_0.gguf?download=true) | Q8 language model; measured decision, chat, and vision setup | 8.01 GB / 7.46 GiB |
| [Winnow-E4B-BF16.gguf](https://huggingface.co/EldanRing/Winnow-E4B/resolve/main/gguf/Winnow-E4B-BF16.gguf?download=true) | Higher-precision language model | 15.05 GB / 14.02 GiB |
| [mmproj-Winnow-E4B.gguf](https://huggingface.co/EldanRing/Winnow-E4B/resolve/main/gguf/mmproj-Winnow-E4B.gguf?download=true) | F16 vision projector | 990 MB / 0.922 GiB |

Choose Q8 for a smaller memory footprint, or BF16 for higher-precision weights. Add the matching projector for images.

Use an exact target filename or a [Winnow preset](docs/RUNTIME-PROFILES.md). [Checksums](https://huggingface.co/EldanRing/Winnow-E4B/blob/main/SHA256SUMS) identify every file.

## Running the model

The [quickstart](https://github.com/EldanRing/winnow-inference/blob/main/docs/QUICKSTART.md) covers installation, downloads, and launch commands. The server provides `/v1/systemone` for decisions and `/v1/chat/completions` for ordinary chat, streaming, and images.

**F16 is the default and recommended target K/V cache.** Cache precision is separate from the GGUF weight format; use `--cache q8_0` for an explicit Q8 override. The pinned MTP assistant uses the target's shared K/V cache.

| Question type | Input | Result |
|---|---|---|
| `noul` | A yes/no question | Probability of `true` |
| `choice` | Named options with optional descriptions | Selected option, probabilities, confidence |
| `score` | Ordered levels | Level probabilities, expected score, confidence |

Questions share a state prefill and can reuse a cached prefix. Candidate probabilities are conditional on the supplied options; confidence measures concentration, not guaranteed correctness.

### Measured performance

On an RTX 5070 Ti 16 GB, the matched 8K text workload delivered **147.8 decisions/s** for E4B Q8 versus **67.2 decisions/s** for 12B Q8 at 64 questions per request. These are medians of ten warm requests with Q8 weights and KV cache; results depend on workload and batching.

| Q8 configuration | Observed peak device memory | Suggested GPU capacity |
|---|---:|---:|
| 8K text-only decision workload | 8.56 GiB | 10 GB+ VRAM |
| 64K vision and chat operational probes | 10.72 GiB across the probes | 12 GB+ VRAM |

Capacity suggestions are inferred from measurements. Six cached questions over a 65,023-position image state took **129–131 ms** after an **11.59 s** first request. A separate 62,431-position multimodal prompt generated 512 tokens at **71.2 tokens/s**. See [timing and memory details](docs/BENCHMARKS.md#same-gpu-latency-and-memory).

### Calibration and adaptive reasoning

For the direct text profile, Q8's fitted `winnow.temperature` is **1.2574172017327816**, based on 778 calibration questions. BF16's separately fitted value is **1.3331553765162731**. Temperature changes probabilities without changing the winning option; the values belong to their respective runtime profiles.

Direct decisions are the default. Adaptive reasoning adds generated context before rescoring a question. A calibrated 50:50 evaluation improved Brier **0.524807 → 0.481402** and NLL **1.591870 → 1.385153**, while source-equal agreement changed **+0.083 pp**. This profile combines calibration with generated context.

A separate historical uncalibrated recipe gained on Jev and Kev but lost **57 Typed teacher agreements** (1,447 → 1,390/2,000), for a net loss of five across all 3,277 decisions. Mean direct HTTP time inside that workflow was **37 ms**; the full router averaged **234 ms**, including bypasses. [Reasoning evaluations](docs/REASONING.md) give the policies and timing definitions.

MTP separately drafts ordinary chat tokens with the matching [E4B assistant](https://huggingface.co/EldanRing/Winnow-E4B/resolve/main/gguf/Gemma-4-E4B-IT-Assistant-BF16.gguf?download=true). The 8K Q8 vision+MTP4 preset sampled **10,197 MiB** peak memory. Direct serving needs no assistant. See [runtime profiles](docs/RUNTIME-PROFILES.md) and [assistant attribution](docs/assistants/README.md).

## Training and limitations

Winnow-E4B is a rank-32, alpha-64 LoRA fine-tune. Language tensors were trained; vision and audio modules remained frozen. The adapter was merged in FP32 before Q8_0 and BF16 export. The private training set combines contrastive synthetic decisions, verified labels, teacher distributions, semantic tasks, and targeted hard cases. Training, development, calibration, and reserved tests were separated; the training data and pipeline are not released.

## Credits and license

Winnow-E4B is an independent fine-tune by EldanRing of Google DeepMind's [Gemma 4 E4B IT](https://huggingface.co/google/gemma-4-E4B-it), released under [Apache 2.0](https://ai.google.dev/gemma/docs/gemma_4_license). See [LICENSE](https://huggingface.co/EldanRing/Winnow-E4B/blob/main/LICENSE) and [NOTICE](https://huggingface.co/EldanRing/Winnow-E4B/blob/main/NOTICE).

The separate inference code builds on [llama.cpp](https://github.com/ggml-org/llama.cpp) and preserves its MIT license. Jev-style describes the typed-decision interface; Winnow is not affiliated with or endorsed by TypeSafe, Google, or llama.cpp.