# Changelog

Revision numbers identify published model states independently of internal run
names. Working run names are retained only in the detailed training provenance.

## r008 - Pollock 1.6 (in preparation)

- Replaced the 3,200-wide GELU FFN with a 2,144-wide SwiGLU FFN while retaining
  16 layers, width 768, 12 attention heads, learned absolute positions, the
  2,048-token context, and the 12,288-entry tokenizer.
- Increased native trainable parameters from 127,427,328 to 127,820,544, leaving
  179,456 parameters below the 128M ceiling.
- Kept the pinned `SlayerLab/minimal-en-corpus-5b` revision, deterministic
  shuffled-window loader, 491,520-target effective batch, Muon/AdamW optimizer
  partition, and complete validation-split evaluation.
- Added a self-contained Transformers SwiGLU implementation and checkpoint
  converter with native/Transformers logit-parity verification.
- Completed the two-pass base run at update 21,959. Its checkpoint and benchmark
  bundle are retained as an E2 baseline, not as the final R008 release.
- Stopped the initial E3-E4 continuation at update 34,520 after the inherited
  R007-derived learning-rate floors proved too high for the SwiGLU continuation:
  both fixed train-eval and full-validation loss degraded while parameter norm
  fell by about 6.5%.
- Forked the verified E2 checkpoint into the final R008 continuation with fixed
  learning rates reduced by 10x, from Muon `1e-3` to `1e-4` and AdamW `3e-5`
  to `3e-6`, while retaining weight decay `1e-2`.
- Stopped the final continuation intentionally at update 38,140 after validation
  plateaued and selected the best checkpoint at update 32,000, after
  15,728,561,562 target presentations, with full-validation loss 2.4507743919.
- Converted the selected checkpoint with exact native/Transformers logit parity;
  `model.safetensors` has SHA-256
  `1c1002909119b4913841ee29efd44bd86d62d81f2ae27de01602b64d4af0dbec`.
- Completed and checksum-verified the final standard model-card, Tiny-ML, Open
  SLM FP32/BF16, ArithMark-3, and fixed-inference suites. The canonical Open SLM
  FP32 Intelligence Index is 13.4419562655 and WikiText-2 word perplexity is
  34.8189798815.