Add evaluated step-438 BF16 GGUF

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 gguf/Winnow-E4B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/Winnow-E4B-BF16.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -38,13 +38,22 @@ quality, vision and 64K context have not been qualified for this E4B artifact.
 | Download | Use | File size |
 |---|---|---:|
 | [Winnow-E4B-Q8_0.gguf](gguf/Winnow-E4B-Q8_0.gguf) | **Q8_0; selected and measured deployment artifact** | 8.01 GB / 7.46 GiB |
+| [Winnow-E4B-BF16.gguf](gguf/Winnow-E4B-BF16.gguf) | Optional higher-precision text model; Q8 remains recommended on 16 GB GPUs | 15.05 GB / 14.02 GiB |
 
 The GGUF contains the merged language weights, tokenizer and chat template. It
-needs no separate LoRA adapter or conversion. This repository currently provides
-the selected **Q8_0 model only**. The original Gemma 4 E4B architecture includes
+needs no separate LoRA adapter or conversion. Both files come from the same
+selected step-438 merge. The original Gemma 4 E4B architecture includes
 multimodal components, but an E4B projector is not packaged here and this release
 does not claim tested image input. Verify downloads with [SHA256SUMS](SHA256SUMS).
 
+The BF16 GGUF passed 8K text-decision serving on an RTX 5070 Ti. On 759 held-out
+ordinary and 132 longer decisions, its top-choice accuracy was essentially the
+same as Q8. Its separately calibrated probability loss was slightly worse and
+it used about **15.0 GiB of GPU memory** versus **8.6 GiB** for Q8, with little
+headroom on a 16 GB card. BF16 has **not** been run through the public suites
+or long-context/vision profile reported for Q8. See the
+[paired format check](docs/BENCHMARKS.md#selected-step-438-bf16-versus-q8).
+
 ## Decision quality
 
 All numbers below are Q8 GGUF results on the same frozen questions. The matched
@@ -91,9 +100,9 @@ generation speeds are not characterized by this table.
 ## GGUF packaging
 
 The selected step-438 language adapter was merged into the pinned Gemma 4 E4B IT
-base as `FP32(base) + FP32(B @ A) × alpha/rank`, then converted to Q8_0 with a
-pinned llama.cpp converter. This avoids rounding the learned update to BF16
-before quantization. The artifact is a complete merged model; the original
+base as `FP32(base) + FP32(B @ A) × alpha/rank`, then converted to Q8_0 or BF16
+with a pinned llama.cpp converter. The FP32 merge avoids rounding the learned
+update to BF16 before conversion. Each artifact is a complete merged model; the original
 adapter and private training checkpoint are not needed for inference.
 
 ## Training
@@ -109,7 +118,7 @@ released GGUF contains the merged weights.
 | LoRA rank / alpha | 32 / 64 |
 | Adapted weights | 294 language tensors; vision and audio modules frozen |
 | Targeted continuation | Selected step 438; effective batch 16; up to 8,192 input tokens |
-| Export | FP32 adapter merge, then Q8_0 GGUF |
+| Export | FP32 adapter merge, then Q8_0 or BF16 GGUF |
 
 **The training dataset is private and is not released.** It combines synthetic
 contrastive decisions, verified labels, teacher distributions where appropriate,
@@ -129,7 +138,9 @@ correctness; calibration quality varies by task and length. On the combined
 already-opened reserved test panels, E4B's calibrated proper loss improved over
 the prior E4B checkpoint while labeled accuracy was unchanged.
 
-Only text decisions at 8K context were measured in the matched release suite.
+BF16's separately fitted decision temperature is **1.333155** on the same 778
+reserved calibration decisions. The Q8 temperature above should not be applied
+to BF16. Only text decisions at 8K context were measured in the matched release suite.
 The model can be loaded by a chat-capable inference engine, but general chat
 quality and multimodal behavior have not been evaluated for this release.
 

SHA256SUMS

@@ -1 +1,2 @@
 840e3f50e5a9c218727f44e121d1b37cc9e2c3b318c8eb422ba6ef2e27b618a2  gguf/Winnow-E4B-Q8_0.gguf
+53c3e504c061f6a7ce84cb6240d427ac6c630850a2cf93f3c2ee980c8136e7e7  gguf/Winnow-E4B-BF16.gguf

docs/BENCHMARKS.md

@@ -1,6 +1,6 @@
-# Winnow-E4B Q8 evaluation
+# Winnow-E4B evaluation
 
-The published artifact is the selected **targeted step-438 Q8_0 GGUF**, SHA-256
+The primary deployment artifact is the selected **targeted step-438 Q8_0 GGUF**, SHA-256
 `840e3f50e5a9c218727f44e121d1b37cc9e2c3b318c8eb422ba6ef2e27b618a2`.
 The reference is the released Winnow-12B Q8 GGUF. The four release suites ran
 sequentially through the same frozen HTTP runner on one RTX 5070 Ti. They used
@@ -81,10 +81,40 @@ the shared temperature; calibration should be checked on a user's own task.
 
 - No data or answers from these benchmark suites are distributed in this model
   repository. The private training dataset is not distributed either.
-- Native BF16 and GGUF Q8 probability outputs are close in aggregate but not
+- Native BF16 safetensors and GGUF Q8 probability outputs are close in aggregate but not
   exactly equal. The selected Q8 export was evaluated as its own artifact.
 - General chat quality, image input and populated 64K context were not tested
   for this E4B release. The reported GPU memory is an 8K text measurement.
 - Synthetic and public benchmark mixes do not establish real-world accuracy
   for every downstream application. The figures above should not be averaged
   into a single score.
+
+## Selected step-438 BF16 versus Q8
+
+The optional BF16 GGUF was built from a regenerated FP32 merge whose 19 shard
+hashes exactly matched the Q8 export's saved merge receipt. Both formats were
+served on the RTX 5070 Ti using the same 8,192-token text-only profile, Q8 KV
+cache, four decision branches and selected answer-token projection. BF16 passed
+all 891 held-out test requests. A separate temperature was fitted for each
+format on the same 778 reserved calibration questions before scoring tests.
+
+| Held-out measure | Q8_0 | BF16 |
+|---|---:|---:|
+| Ordinary accuracy, 759 questions | 74.52% | 75.01% |
+| Longer-question accuracy, 132 questions | 53.03% | 53.03% |
+| Ordinary proper loss, calibrated | **0.4339** | 0.4386 |
+| Longer-question proper loss, calibrated | **0.9485** | 0.9582 |
+| Fitted temperature | 1.257417 | 1.333155 |
+| Ordinary HTTP latency, median | **38.3 ms** | 41.4 ms |
+| Longer-question HTTP latency, median | **597 ms** | 761 ms |
+| GPU memory observed while serving | **8,769 MiB** | 15,346 MiB |
+
+The 699 ordinary labeled questions had six BF16-only correct answers and four
+Q8-only correct answers. The 132 longer labeled questions had four wins for
+each format. These small differences do not establish BF16 as more accurate.
+BF16's calibrated probability loss was slightly worse on both splits. The HTTP
+latencies were collected in separate sequential runs, so they are indicative
+for this profile rather than an isolated kernel-speed comparison. BF16 leaves
+less than 1 GiB of measured VRAM headroom on the 16 GB card at 8K; Q8 remains
+the recommended 5070 Ti format. BF16 was not evaluated on the public suites,
+with an image projector, or at a populated 64K context.

docs/QUICKSTART.md

@@ -1,6 +1,7 @@
 # Run Winnow-E4B Q8
 
-This private repository currently contains a **text-only** Q8 GGUF. The
+This private repository contains text-only-tested Q8 and BF16 GGUFs. Q8 is the
+recommended format for a 16 GB GPU. The
 measured RTX 5070 Ti profile uses 8,192 context positions, Q8 KV cache and four
 decision branches. [Winnow's inference server](https://github.com/EldanRing/winnow-inference)
 provides the typed `/v1/systemone` API and regular chat.
@@ -26,7 +27,7 @@ Authenticate with Hugging Face while this model repository is private. From the
 hf download EldanRing/Winnow-E4B \
   gguf/Winnow-E4B-Q8_0.gguf SHA256SUMS \
   --local-dir models/Winnow-E4B
-(cd models/Winnow-E4B && sha256sum --check SHA256SUMS)
+(cd models/Winnow-E4B && sha256sum --check --ignore-missing SHA256SUMS)
 ```
 
 The 8.01 GB model file is self-contained for text use; it needs no separate
@@ -75,3 +76,9 @@ set. Omit it to use raw candidate probabilities. For more request types and chat
 see the [API documentation](https://github.com/EldanRing/winnow-inference/blob/main/docs/API.md).
 Image input requires a matching projector and remains outside this release's
 validated E4B profile.
+
+To use BF16 on a GPU with sufficient memory, download
+`gguf/Winnow-E4B-BF16.gguf` and substitute that path in the serve command.
+Use `1.3331553765162731` as the decision temperature fitted for BF16, or omit
+the temperature for raw probabilities. BF16 uses about 15.0 GiB on the tested
+8K text profile and has not shown a quality advantage over Q8.

gguf/Winnow-E4B-BF16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:53c3e504c061f6a7ce84cb6240d427ac6c630850a2cf93f3c2ee980c8136e7e7
+size 15053096992

release-manifest.json

@@ -13,6 +13,11 @@
       "path": "gguf/Winnow-E4B-Q8_0.gguf",
       "bytes": 8005437472,
       "sha256": "840e3f50e5a9c218727f44e121d1b37cc9e2c3b318c8eb422ba6ef2e27b618a2"
+    },
+    "bf16": {
+      "path": "gguf/Winnow-E4B-BF16.gguf",
+      "bytes": 15053096992,
+      "sha256": "53c3e504c061f6a7ce84cb6240d427ac6c630850a2cf93f3c2ee980c8136e7e7"
     }
   },
   "decision_calibration": {
@@ -21,6 +26,12 @@
     "calibration_rows": 778,
     "scope": "text decisions; 8K-context deployment"
   },
+  "bf16_decision_calibration": {
+    "artifact": "bf16",
+    "temperature": 1.3331553765162731,
+    "calibration_rows": 778,
+    "scope": "text decisions; 8K-context deployment"
+  },
   "measured_profile": {
     "gpu": "NVIDIA RTX 5070 Ti",
     "context": 8192,