# Winnow-12B usage draft

Weights and the public inference repository URL are pending. The commands below
describe the planned release filenames; they cannot download weights yet.

## Download the Q8 model

Once the artifacts exist:

```sh
hf download EldanRing/Winnow-12B \
  gguf/Winnow-12B-Q8_0.gguf gguf/mmproj-F16.gguf \
  --local-dir models/Winnow-12B
```

Private-repository downloads require an authorized Hugging Face login. The vision
projector is optional for text-only use. Verify the files against the final
manifest before loading them.

## Start the inference server

From a built Winnow inference checkout, the 5070 Ti launch profile is:

```sh
python3 scripts/serve.py \
  --model models/Winnow-12B/gguf/Winnow-12B-Q8_0.gguf \
  --mmproj models/Winnow-12B/gguf/mmproj-F16.gguf \
  --context 65536 --decision-parallel 4 --cache q8_0
```

Final-model confirmation is pending. The server binds to localhost by default.
Configured context includes prompt formatting, images, questions and chat output.

## Typed decisions

```sh
curl http://127.0.0.1:8091/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Winnow-12B",
    "state": {"paid": true, "priority": "high"},
    "questions": {
      "paid": {"type": "noul", "instructions": "Has the customer paid?"},
      "route": {
        "type": "choice",
        "instructions": "Which priority is recorded?",
        "criteria": {"low": null, "high": null}
      },
      "rating": {
        "type": "score",
        "instructions": "How urgent is the recorded priority?",
        "criteria": ["not urgent", "moderately urgent", "very urgent"]
      }
    }
  }'
```

`noul` returns a probability of true. `choice` returns an option and its
distribution. `score` returns an expected zero-based category index and its
distribution. Question batch size and resident parallel branches are different:
larger batches run in waves.

## Chat

```sh
curl http://127.0.0.1:8091/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"Winnow-12B","messages":[{"role":"user","content":"Explain how prefix caching helps a local decision API."}],"max_tokens":4096,"stream":true}'
```

Chat uses the original template and the full vocabulary. Vision chat uses
OpenAI-style image content with the matching projector loaded. If server
authentication is configured, add the bearer authorization header to requests.
The detailed build, vision and scheduling documentation will live in the
inference source repository.