---
base_model: google/gemma-4-12B-it
base_model_relation: finetune
license: apache-2.0
pipeline_tag: text-generation
tags:
  - winnow
  - gemma4
  - typed-decisions
  - local-inference
  - vision
---

# Winnow-12B

**Private release draft. Final checkpoint selection and weight export are pending.**
This repository currently contains release documentation, not runnable model weights.

Winnow-12B is a LoRA fine-tune of Google DeepMind's Gemma 4 12B for local typed decisions. The accompanying
Winnow inference server combines a Jev-style decision API with ordinary
OpenAI-style chat and vision chat, using the same loaded model.

- **Typed decisions:** shared state with named `noul`, `choice` and `score` questions.
- **Fast inference:** shared-prefix reuse and parallel question branches, with
  candidate logits read directly instead of generating answer text.
- **Chat:** standard `/v1/chat/completions`, including streaming and vision with
  the matching projector. Final-checkpoint chat retention is still being evaluated.
- **Local deployment:** native CUDA on Linux and Metal on Apple Silicon through
  the accompanying llama.cpp-based inference code.

## Planned artifacts

| Artifact | Purpose | Status |
|---|---|---|
| Merged BF16 safetensors and matching configuration/tokenizer | Full merged model | Pending |
| `gguf/Winnow-12B-Q8_0.gguf` | Local quantized inference | Pending |
| `gguf/mmproj-F16.gguf` | Matching vision projector | Pending verification |
| SHA256 checksums and release manifest | Verify exact files and provenance | Pending artifacts |
| Inference source repository | Decision API, chat, build and checks | Prepared locally; URL pending |

The BF16 export will contain the fine-tuning update merged into the base weights.
It will not require loading an extra adapter. The Q8 GGUF will be made from that
same merged checkpoint. The model name is **Winnow-12B**.

See [usage examples](docs/QUICKSTART.md) and the
[benchmark reporting plan](docs/BENCHMARKS.md).

## Training

The underlying base is [google/gemma-4-12B-it](https://huggingface.co/google/gemma-4-12B-it).
The final base revision, adapter rank/alpha/target modules, objective, training
duration and selected checkpoint will be recorded after selection. Training uses
LoRA; merged weights should not be described as full-parameter fine-tuning.

**The training dataset is private and will not be released.** The final card will
describe its size, task coverage, synthetic and teacher contributions, curation,
split construction and known evaluation overlap at a high level.

## Evaluation

Final checkpoint scores are pending. Decision quality, probability calibration,
chat/vision retention and runtime performance will be reported separately.
Results will identify the exact artifact, benchmark version, sample count and
prompt/probability settings. Earlier candidate scores are not scores for this release.

The runtime has previously been tested with 64K context plus vision on a 16 GB
RTX 5070 Ti, and with longer context profiles. The final Q8 file will receive its
own confirmation before those measurements are assigned to this release.

## Probability interpretation

Decision probabilities are normalized over the supplied answer options.
Entropy-based confidence measures how concentrated that distribution is; it is
not a measured probability that the answer is correct. Raw temperature defaults
to 1. Any fitted calibration will name its fitting split and held-out evaluation.

## Release limitations

Long-context fitting and simple retrieval checks do not establish general
long-context reasoning quality. Mixed chat and decisions share GPU resources;
full-capacity requests may queue or switch contexts. The decision API follows
Jev-style typed requests, without claiming identical hosted-service behavior.

## Credits and licensing

- **Base model:** Google DeepMind's [Gemma 4 12B](https://huggingface.co/google/gemma-4-12B-it),
  released under [Apache 2.0](https://ai.google.dev/gemma/docs/gemma_4_license).
  Winnow adds task-specific LoRA post-training; the selected update will be merged
  into BF16 and converted to Q8 GGUF. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
- **Inference runtime:** [llama.cpp](https://github.com/ggml-org/llama.cpp) by
  Georgi Gerganov and contributors, under the MIT license. The separate inference
  repository preserves its license and documents the Winnow modifications.
- **Winnow:** fine-tuning, evaluation and the typed-decision integration by EldanRing.

This is an independent derivative, not an official Google or llama.cpp release.
The final artifact manifest will identify modified weight files and their source
revision. Projector provenance and any additional source notices will be checked
before uploading the weights. The inference code's MIT license is separate from
the model's Apache 2.0 license.