AbstractPhil commited on
Commit
f18be3e
Β·
verified Β·
1 Parent(s): 2c20ac4

TECHNICAL companion for the 2s era (verified code-side + hub-side, 2026-09-04)

Browse files
Files changed (1) hide show
  1. TECHNICAL_2s.md +1064 -0
TECHNICAL_2s.md ADDED
@@ -0,0 +1,1064 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ <!-- VERIFIED 2026-09-04: code-side (wf_15fc8b16) and hub-side (wf ft2-finish) passes applied. -->
2
+
3
+ # alephllm-mini-beatrix-training β€” Technical Companion (the 2s era: the full-splat craft and its softmax twin)
4
+
5
+ Companion to *Twinning Beatrix: A Full-Splat Byte Model, Its Softmax
6
+ Control, and What Reaches an Image Generator* (AbstractPhil with Claude
7
+ Fable 5 and Claude Fable 5.1, September 2026). It covers two crafts and
8
+ one packaged model: `mini-beatrix-2s/` and `mini-beatrix-2s-control/`
9
+ under AbstractPhil/alephllm-mini-beatrix-training, and the AutoModel at
10
+ AbstractPhil/mini-beatrix-2s; the dates run from the first launch
11
+ (run0, 2026-08-26) through the control's endpoint (2026-09-03) and the
12
+ claim audit that graded it (2026-09-04). It is written for a reader who
13
+ will load the checkpoints and read the reports. It is not the
14
+ mini-beatrix-1 life record (`TECHNICAL.md`, which stays as it is), not
15
+ the article's argument, and not a manual for the side campaigns that
16
+ share the repo root (`reports/v2/{battery,wide_moe,aflow,bcd}` appear
17
+ only as a catalogue line). Every number carries its pointer: `file:line`
18
+ means E:/mirel/AlephLLM/geolip/alephllm/ at HEAD 4f4b7d6 (package
19
+ 0.8.7, the last commit of the 2s era); hub paths are relative to the
20
+ craft prefix; journal paths are relative to the claude-mind repo. Where
21
+ the journal and the code disagree, the code wins and the disagreement
22
+ is reported in place. Stack-side detail lives in the
23
+ [AlephLLM companion](https://github.com/AbstractEyes/alephllm), which
24
+ covers the 0.7.0 mechanisms only; the 0.8.x organs (specials, driver
25
+ contract, revival, gauge fallback, twin isolation) are carried here.
26
+
27
+ ## The two crafts at a glance
28
+
29
+ | | mini-beatrix-2s | mini-beatrix-2s-control | mini-beatrix-1 |
30
+ |---|---|---|---|
31
+ | d_model / layers / heads | 1024 / 20 / 16 | 1024 / 20 / 16 | 768 / 16 / 12 |
32
+ | attention | CausalSplatHUB in all 20 blocks | CausalSDPA (16 Γ— 64) in all 20 | 13 sdpa + 3 hubs (layers 4, 9, 14) |
33
+ | hub books Γ— anchors @ D | 4 Γ— 64 @ 128 (supply 0.5Γ—) | β€” | 1 Γ— 512 @ 32 (supply 16Γ—) |
34
+ | banks | 3 experts @ ff 1024 | 3 @ 1024 | 3 @ 768 (full width) |
35
+ | head (K @ D) | 256 @ 256, revived at 24,860, address frozen after | 256 @ 256, born null, never elected | 512 @ 32, never elected (W_s = 0.0 at ship) |
36
+ | context / vocab | 4096 / 256 byte-trigram | 4096 / 256 | 2048 / 256 |
37
+ | params | 237,125,949 | 257,442,109 (+20,316,160, all attention) | 112,524,593 |
38
+ | tokens per step | 16 Γ— 4 Γ— 4096 = 262,144 | 262,144 | 48 Γ— 3 Γ— 2048 = 294,912 |
39
+ | steps / tokens | 61,422 / 16,101,408,768 | 61,422 / 16,101,408,768 | 88,508 / 26,102,071,296 |
40
+ | wall-hours / hosts | 78.33 on one RTX PRO 6000 | 49.98 on three hosts | 70.45 |
41
+ | final val bpb (report ledger) | **1.1097** | **2.8846** | 1.121 |
42
+ | optimizer | Muon 2e-2 + pure Adam 3e-4, flat after 200 | same | same |
43
+ | governor | minsep 45Β°, every 8 steps, from birth | inherited (head book only) | off |
44
+ | package at birth β†’ end | 0.8.1 β†’ 0.8.6 (revival under 0.8.2) | 0.8.7 throughout | cc89a87 β†’ 0.6.0 (pre-specials) |
45
+
46
+ Shape and knobs: `presets.py:191-203` (2s), `:214-235` (the `-control`
47
+ derivation and the head_addr_frozen override), `:155-160` (mb1).
48
+ Parameter counts were computed by constructing each preset on the meta
49
+ device (`alephlm.py:218-224` `param_count`): 2s = embed 4,982,784 +
50
+ 20 blocks Γ— 11,574,275 (hub attention 3,178,496 + bank 8,391,683 +
51
+ norms 4,096) + final norm 2,048 + head 655,617; the control's block
52
+ swaps the hub's 3,178,496 for sdpa's 4,194,304 (qkv 3072Γ—1024 + o
53
+ 1024Γ—1024, no biases; `attention.py:24-32`). Tokens per step is
54
+ `micro_batch Γ— grad_accum Γ— context` (`trainer.py:186`); 61,422 Γ—
55
+ 262,144 lands on both manifests' `tokens_seen` exactly. The mother's
56
+ numbers are the exemplar's (`TECHNICAL.md`, `mini-beatrix-1/manifest.json`).
57
+
58
+ ## Layout
59
+
60
+ ```
61
+ mini-beatrix-2s/
62
+ manifest.json # 14 phases, 142 checkpoint entries, notes
63
+ checkpoints/step_XXXXXXXX.safetensors # 46 bf16 (2000-cadence + every boundary and cap)
64
+ checkpoints/step_00024860_revived.safetensors # fp32, the post-BOUNDARY-WRITE head
65
+ checkpoints/fp8/ # 20 e4m3 ship-only copies
66
+ resume/latest.pt # fp32 model + Muon/Adam + stream + RNG + manifest
67
+ resume/boundary_<phase>_step<N>.pt # 13 immutable boundary archives
68
+ resume/pre_revival_step24860.pt # the state the revival overwrote
69
+ reports/v2/ # 13 boundary + 3 session_cap + revival + 4 cultivation
70
+ runs/ # 5 TensorBoard event files (one per session)
71
+ archive/run0-nospecials/ # the pre-specials first attempt, whole
72
+ beatrix_v2_colab.ipynb # the driver
73
+
74
+ mini-beatrix-2s-control/
75
+ manifest.json # identical phases and budgets; 136 entries
76
+ checkpoints/step_XXXXXXXX.safetensors # 44 bf16 (incl. the 22,854 cap)
77
+ checkpoints/fp8/ # 20 e4m3 ship-only copies
78
+ resume/latest.pt resume/boundary_*.pt # latest + 13 boundary archives
79
+ reports/v2/ # 13 boundary + 2 session_cap (22,854; 53,792)
80
+ runs/ # 6 event files (two are 88-byte aborted starts)
81
+ ```
82
+
83
+ The control has no notebook file and no AutoModel packaging; these
84
+ checkpoints are its weights. Files: bf16 475,876,922 B (2s) and
85
+ 515,164,234 B (control); fp8 238,219,518 / 257,813,142 B; resume
86
+ archives ~1.904 / ~2.064 GB each; the revived fp32 file 951,692,996 B
87
+ (hub headers read by ranged GET; `checkpoint.py:102-128` for the two
88
+ export rules).
89
+
90
+ ## Architecture in detail
91
+
92
+ **The block.** Pre-norm, two residual branches: `x = x + attn(n1(x))`,
93
+ `x = x + bank(n2(x))` (`alephlm.py:37-70`). `attn` is CausalSplatHUB
94
+ when the layer index is in `hub_layers`, else CausalSDPA
95
+ (`alephlm.py:41-50`); every block carries an AnchoredBank
96
+ (`alephlm.py:51`). `hub_ckpt` (0 on 2s: the retained scan fits the
97
+ 96 GB card at 237M, `presets.py:184-198`) would wrap the branches in
98
+ activation checkpointing (`alephlm.py:60-68`). The model is embedding β†’
99
+ 20 blocks β†’ final LayerNorm β†’ DualHead (`alephlm.py:81-96`); `forward`
100
+ takes pre-shifted `targets` (ours) or HF-style `labels` (shifted
101
+ internally) and ignores `attention_mask` deliberately
102
+ (`alephlm.py:98-124`).
103
+
104
+ **Trigram embedding and positions.** `e_t = emb0[x_t] + emb1[x_{t-1}] +
105
+ emb2[x_{t-2}] + pos[t]`; the shift tables have a 257th PAD_ROW so the
106
+ two positions shifted in at the sequence start never read a legal byte
107
+ (`embedding.py:19-34`; the pad law β€” padding with a real byte starved
108
+ address consumption, measured +.05..+.11 on repair). `pos` is a learned
109
+ absolute table `(1, 4096, 1024)` (`embedding.py:28`); there is no RoPE.
110
+
111
+ **CausalSplatHUB β€” the address.** Each book is an `AlephAddress`: K unit
112
+ anchors in D dims (`address.py:24-36`, with a `home` buffer holding the
113
+ birth copy for drift). The read is a cosine at temperature,
114
+ `u_k = cos(xΜ‚, Γ’_k)/Ο„` with Ο„ = 0.1 (`address.py:38-40`); the oriented
115
+ form is the 2K-softmax over the axes `(+a_k, βˆ’a_k)`, and the signed read
116
+ is its half-difference `w_k = sinh(u_k)/Ξ£_j cosh(u_j)` β€” inhibition
117
+ (negative w) is first-class; there is no softmax over choices, no
118
+ argmax, no top-k (`address.py:1-13,42-66`). The hub's q and k frames are
119
+ `Linear(d, D)` per book, orthogonal-init; v and o are shared `Linear(d,
120
+ d)` (`attention.py:64-79,106-109`). Construction warns when K > 2D β€” the
121
+ supply law from ROUND 5e and the mini-beatrix-1 hub census
122
+ (`attention.py:85-92`); 2s runs 64 @ 128, half of the lawful ceiling.
123
+
124
+ **CausalSplatHUB β€” books and budget.** With `n_const = 4` the four books
125
+ each own a codebook and q/k frames (`attention.py:64-79`); their
126
+ numerators and agreement masses are summed across books before the one
127
+ divide β€” composition by budget, never softmax over books
128
+ (`attention.py:245-254`; B4 measured comparative composition at βˆ’.10).
129
+ `n_const = 1` is the v1 layout with the same state-dict keys
130
+ (`attention.py:93-101`), which is how mini-beatrix-1 checkpoints still
131
+ load.
132
+
133
+ **CausalSplatHUB β€” the exact chunked scan (prefix memory).**
134
+ `_code_cat_qk` computes q and k for all books in one stacked GEMM,
135
+ folds the row-norm and Ο„ into one post-GEMM scale, and takes ONE fused
136
+ softmax over `cat([u, βˆ’u])` β€” the module's docstring names this a
137
+ kernel substitution, not a mechanism change: the softmax that exists is
138
+ over the 2K oriented axes of a single query, never over positions
139
+ (`attention.py:154-212`). `_scan_cat` splits the sequence into chunks of
140
+ C = 256 (16 chunks at ctx 4096), forms per-chunk `(2K Γ— d)` sums, takes
141
+ the exclusive prefix over chunks as a strictly-lower-triangular GEMM
142
+ with fp32 accumulation, adds the within-chunk causal affinity, and
143
+ divides the numerator by the scalar agreement mass `den`, floored at
144
+ `dtype_floor` (`attention.py:139-151,229-281`; `address.py:16-22`: fp16
145
+ flushes 1e-12 to zero, the measured landmine). Exactness is
146
+ chunk-independent (`forward_naive` is the oracle, `attention.py:320`;
147
+ fp reorder ~1.5e-06). Decode carries a constant-size state per book β€”
148
+ `{Ξ£ kβΊβŠ—v, Ξ£ kβ»βŠ—v, Ξ£ k⁺, Ξ£ k⁻}` β€” regardless of length
149
+ (`attention.py:284-318`); the generate path falls back to a sliding
150
+ window once the position table is exhausted because "the hub's
151
+ constant-size state cannot evict" (`alephlm.py:180-201`).
152
+
153
+ **The autocast trap.** `torch.einsum` sits in autocast's PROMOTE
154
+ category and `F.normalize`/`exp`/`softmax` on its fp32 list, so one fp32
155
+ operand dragged the whole scan to fp32 (`attention.py:167`); 0.7.2
156
+ (@2be28dd) cast the operands explicitly. Every earlier "bf16" splat run
157
+ had silently run fp32 scans; sdpa was unaffected β€” so every pre-0.7.2
158
+ splat-vs-sdpa wall-clock comparison favoured sdpa, and accuracy
159
+ verdicts stand (autocast einsum-promote law).
160
+
161
+ **The governor.** `minsep_project_` bounds `|cos(a_i, a_j)| ≀
162
+ cos(ΞΈ_min)` for every anchor pair by an exact symmetric geodesic
163
+ push-apart, largest violation first, projected to ΞΈ_min + 0.5Β°
164
+ (`governor.py:35-80`); `govern_model` collects hub and head codebooks,
165
+ runs one batched `bmm` slack check with one device sync, and projects
166
+ only the violating books (`governor.py:92-137`). It is a
167
+ post-optimizer-step projection, never an auxiliary loss
168
+ (`governor.py:11`); the trainer calls it every 8 steps after `opt.step`
169
+ and accumulates pair-pushes into `governor/hits_cum`
170
+ (`trainer.py:263-267,294-296`; `presets.py:99-105`: ΞΈ = 45Β°). Banks are
171
+ excluded by default (a 3-anchor book never crowds). The docstring
172
+ records the small-rig verdict it shipped on (gov25m: fires only when
173
+ needed, census tail cut 5.4Γ—, fragile-seed +.007, stable-seed +.002;
174
+ `governor.py:19`). Code-wins note: the journal's covariance program
175
+ describes a *metric* governor through the frame W_a; alephllm's is the
176
+ plain projective form on the D-space books, justified in place β€”
177
+ "Hub and head codebooks live natively in their own D-dim address space"
178
+ (`governor.py:14`), since the q/k frames project d β†’ D before the cosine
179
+ read (`address.py:38-40`).
180
+
181
+ **CausalSDPA in the twin.** `qkv = Linear(d, 3d)`, `o = Linear(d, d)`,
182
+ both orthogonal-init and bias-free; `F.scaled_dot_product_attention(q,
183
+ k, v, is_causal=True)` over 16 heads of 64 (`attention.py:24-42`);
184
+ the KV cache grows O(n) (`attention.py:43-61`). No QK-norm, no logit
185
+ softcap.
186
+
187
+ **AnchoredBank.** `Bank(x) = trunk(x) + Ξ£_k w_k(x)Β·Οƒ(g_k)Β·expert_k(x)`
188
+ (`bank.py:3`): a dense trunk `t_out(GELU(t_in(x)))`, three experts
189
+ `(3, 1024, 1024)` with `w_out` zero-initialised (the null path), gates
190
+ at βˆ’3.0, and dispatch `w = addr.signed(x)` through a 3-anchor book in d
191
+ dims. `disable_dispatch` returns the trunk alone β€” bit-identical to the
192
+ dense control at birth (the C6 null); "No balance machinery of any
193
+ kind" (`bank.py:10,41-50`).
194
+
195
+ **DualHead.** `logits = W_h h + Ξ³Β·W_s s(h)`, `s(h) = addr.signed(proj(h))`
196
+ with proj `Linear(1024, 256)`, a 256 @ 256 book, `W_s` zero-initialised
197
+ ("weight-zero: self-starting") and Ξ³ = 1 frozen as an ablation knob
198
+ (`head.py:23-51`). The docstring carries the law that set it: on
199
+ mini-beatrix-1 over 8.4B tokens a gamma-gated branch DEADLOCKS while a
200
+ weight-zero bank elected into +2 bpb (`head.py:8-11`; gate-zero deadlock
201
+ law). `disable_aleph` returns `W_h h` alone β€” the head toggle.
202
+
203
+ **The special-token control plane.** Thirteen ids that no UTF-8 encoder
204
+ can emit β€” `{0xC0, 0xC1} βˆͺ [0xF5, 0xFF]`, the overlong leads and the
205
+ leads past U+10FFFF β€” carry structure inside vocab 256: DOC 0xFF, USER
206
+ 0xFE, MODEL 0xFD, END 0xFC, SYS 0xFB, THINK 0xFA, DATA 0xF9, SEP 0xF8,
207
+ MODE 0xF7 (+1 ASCII payload), CUE 0xF6, ESC 0xF5 (+payload = 256
208
+ extended slots; 0x00 the null guard, 0x01 CANARY), RES0 0xC0, RES1
209
+ 0xC1 (`special_tokens.py:20-34,50-71`; `assert len(SPECIALS) == 13`).
210
+ The proof is exhaustive β€” every codepoint 0..0x10FFFF encoded and
211
+ checked β€” and the Trainer runs it at every byte-craft launch
212
+ (`trainer.py:72-73`). Doctrine: DOC is appended by the packer after
213
+ every document in every phase from birth, identity-gated to the
214
+ byte-trigram tokenizer (`streams.py:59-62,209,355-356`); the chat frame `[SYS] text
215
+ [END] [USER] text [END] [MODEL] text [END]` (`special_tokens.py:112-136`)
216
+ is unforgeable (content is UTF-8) and appears only in the anneal
217
+ (`special_tokens.py:36-44`; chat-in-anneal law). The canary frame moved
218
+ to CUE/SEP because the old QRY=255 collided with DOC
219
+ (`canaries.py:20-29`); `recall_acc` re-baselines at 0.8.0. The trigram
220
+ embedding feeds each special into the next two positions' input
221
+ representations by construction.
222
+
223
+ ## The recipe (pinned from source; unchanged cc89a87 β†’ b472815)
224
+
225
+ - **Split**: every parameter with `ndim ∈ {2, 3}` that is not an
226
+ `nn.Embedding` weight rides Muon; everything else β€” the three byte
227
+ tables, LayerNorms, bank gates, Ξ³, the head bias β€” rides
228
+ `torch.optim.Adam(weight_decay=0.0)` (`optim.py:54-79`; "pure Adam,
229
+ wd=0 β€” never AdamW", `presets.py:82`). Measured split for 2s: Muon 385
230
+ tensors / 236,253,184 numel, Adam 107 / 872,765; the control's Muon
231
+ set is 145 / 256,569,344 with the same Adam set. Code-wins note: the
232
+ `optim.py:4-5,55-58` docstrings say positions ride Adam; the code
233
+ sends `embed.pos` (shape `(1, 4096, 1024)`, ndim 3, not an
234
+ `nn.Embedding`) to Muon (`optim.py:68`), verified on the meta device.
235
+ The journal already records the code's behaviour.
236
+ - **Muon**: 5-step Newton–Schulz (a, b, c = 3.4445, βˆ’4.7750, 2.0315) on
237
+ the momentum buffer after whole-matrix Frobenius normalisation
238
+ `X/(β€–Xβ€–_F + 1e-7)`; plain momentum 0.95, no Nesterov, no shape
239
+ scaling; 3-D stacks orthogonalised per slice (`optim.py:20-51`).
240
+ - **Learning rates**: Muon 2e-2, Adam 3e-4; `lr_scale = (step+1)/200`
241
+ for 200 steps, then 1.0 forever (`optim.py:82-93`, `presets.py:80-83`).
242
+ The tfevents read `train/lr_scale` 0.25/0.50/0.75 at steps 50/100/150
243
+ and 1.000 from step 200 to the last log on both crafts, through both
244
+ anneal phases: the anneal is a diet change, not a decay.
245
+ - **Clip**: `clip_grad_norm_(…, 1.0)` over all parameters; the logged
246
+ `train/grad_norm` is the PRE-clip norm it returns (`trainer.py:244,
247
+ 287`). Because Muon normalises the momentum buffer before
248
+ orthogonalising, its update magnitude is independent of gradient
249
+ scale: clipping never changes the magnitude of a Muon step; it only reweights
250
+ steps inside the momentum buffer. The Adam set (`torch.optim.Adam`,
251
+ wd 0, default eps 1e-8; `optim.py:79`) is per-coordinate
252
+ scale-normalised (`m/sqrt(v)`), so under a persistent clip its step
253
+ magnitude is likewise unchanged and the clip only reweights the two
254
+ moment averages (Muon is clip-blind; Adam is nearly so).
255
+ - **Precision and batch**: fp32 masters, bf16 autocast on CUDA, TF32 on
256
+ (`trainer.py:14-15,65-66,239`); micro-batch 16 Γ— accum 4 Γ— ctx 4096;
257
+ seed 1337 (`presets.py:97,199`); each micro-batch row is 4097 ids
258
+ scored at 4096 positions.
259
+ - **Guards**: a non-finite loss or gradient raises before any optimizer
260
+ step, weights untouched and the resume state not overwritten
261
+ (`trainer.py:250`); the spike flag `loss > 2Β·EMA + 0.5` is logged
262
+ only (`trainer.py:273-275,293`). Neither fired on either craft.
263
+ - **Cadences**: log 50, health census 500, eval and checkpoint 2000,
264
+ fp8 every 5th checkpoint, TB upload 1000, val 262,144 tokens, 128
265
+ canary episodes (`presets.py:88-96`).
266
+ - **Absent by design**: no QK-norm, no logit softcap, no z-loss, no
267
+ weight decay, no dropout, no RoPE, no LR decay in any phase.
268
+
269
+ ## The diet (all on one gauge: fineweb-edu holdout bpb)
270
+
271
+ | phase | dataset | budget | β†’ step | 2s bpb | control bpb | note |
272
+ |---|---|---|---|---|---|---|
273
+ | warmup_wikitext | wikitext-103 | 0.30B | 1,145 | 2.3626 | 2.1056 | sdpa led by 0.257; protocol pre-registered here |
274
+ | fineweb_main | fineweb-edu | 5.0B | 20,219 | **1.1172** | 1.1763 | the crossover; hubs +2.22 / banks +3.55 |
275
+ | fineweb_extended | fineweb-edu | 10B | β€” | β€” | β€” | DEFERRED, unspent on both |
276
+ | curriculum_s0 | curriculum-s0 | 0.7B | 22,890 | 1.2522 | 1.5047 | |
277
+ | REVIVAL | β€” | β€” | 24,860 | 1.2768 β†’ 1.2339 (op gauge) | β€” | BOUNDARY-WRITE; `pre_revival_step24860.pt` archived |
278
+ | curriculum_s1 | curriculum-s1 | 0.7B | 25,561 | 1.2544 | 1.3917 | first live head toggle in the lineage, +0.0258 |
279
+ | curriculum_s2 | curriculum-s2 | 1.0B | 29,376 | 1.2177 | 1.3938 | 2s grad-norm curriculum+anneal max 0.619 @26,750 |
280
+ | curriculum_s3 | curriculum-s3 | 1.2B | 33,954 | 1.2318 | 1.5317 | |
281
+ | curriculum_s4 | curriculum-s4 | 1.2B | 38,532 | 1.2356 | 1.5689 | both 2s mechanisms above +3.0 |
282
+ | curriculum_s5 | curriculum-s5 | 1.0B | 42,347 | 1.2504 | 2.1762 | control grad-norm median 126 |
283
+ | curriculum_s6 | curriculum-s6 | 0.8B | 45,399 | 1.2683 | 3.1940 | control max 41,070 @44,600 |
284
+ | curriculum_s7 | curriculum-s7 | 1.4B | 50,740 | 1.1883 | 3.1201 | |
285
+ | curriculum_s8 | curriculum-s8 | 0.8B | 53,792 | *1.0737* (synthetic gauge) / **1.1911** (cap) | 3.0810 | 2s row STRUCK; true ledger = `session_cap_step53792` |
286
+ | anneal_nochat | anneal-nochat | 1.0B | 57,607 | 1.1153 | 2.9270 | the pre-chat baseline; probe mean .389 |
287
+ | anneal_mix | anneal-mix | 1.0B | 61,422 | **1.1097** | **2.8846** | run best; probe mean .389 == pre-chat |
288
+
289
+ Phases and budgets: `presets.py:132-146` (the preset plan carries
290
+ anneal_mix at 2B deferred; the notebook's C6 cell rewrote it to 1B and
291
+ inserted anneal_nochat before it β€” "two-phase anneal activated: nochat
292
+ 1B then chat 1B (plan of record 2026-08-31)" in both manifests' notes; the
293
+ hub copy of `beatrix_v2_colab.ipynb`, uploaded 2026-08-28T23:00Z with cells
294
+ C1–C5 and pinned to `geolip-alephllm @2a902b7` (0.8.1), predates the
295
+ amendment and carries no C6 cell),
296
+ `curriculum.py:657-663` (STAGE_TOKENS), `curriculum.py:666-689`
297
+ (append at runtime). Manifest order is not chronology: the S0–S8 phases
298
+ are listed after the anneals but ran before them, because
299
+ `current_phase()` activates the first `planned` phase and deferred
300
+ phases are invisible until flipped (`manifest.py:33-41`); count back
301
+ from the totals (16,101,408,768 βˆ’ anneal_mix's 1,000,079,360 lands on
302
+ step 57,607) rather than reading the list.
303
+
304
+ **The mixes** (`curriculum.py:614-655`; weights sum to 1): s0
305
+ tinystories .74, simple-wiki .20, aochildes .01, recall-synth .03,
306
+ counting-synth .02 Β· s1 perspective-synth .30, tinystories .28,
307
+ simple-wiki .20, cosmo-young .12, concept-synth .04, recall-synth .03,
308
+ siqa-narrative .02, aochildes .01 Β· s2 concept-synth .30, cosmo-young
309
+ .30, simple-wiki .12, tinystories .10, definitions .08, wikipedia-en
310
+ .07, recall-synth .03 Β· s3 rulechain-synth .32, proofwriter-prose .22,
311
+ simple-wiki .12, tinystories .10, wikipedia-en .08, cosmo-young .08,
312
+ concept-synth .04, recall-synth .03, babi-prose .01 Β· s4 arith-synth
313
+ .35, cosmo-young .32, simple-wiki .12, tinystories .08, rulechain-synth
314
+ .06, counting-synth .04, recall-synth .03 Β· s5 causal-synth .32,
315
+ atomic-flicker .25, tinystories .12, simple-wiki .10, gutenberg .08,
316
+ cosmo-young .07, recall-synth .05, siqa-narrative .01 Β· s6
317
+ tryfail-synth .32, tinystories .22, arith-synth .13, cosmo-young .10,
318
+ simple-wiki .10, causal-synth .08, recall-synth .05 Β· s7 tinystories
319
+ .10, cosmo-young .10, rulechain-synth .10, arith-synth .10, fineweb-good
320
+ .10, simple-wiki .08, atomic-flicker .08, causal-synth .08,
321
+ tryfail-synth .08, proofwriter-prose .08, recall-synth .05,
322
+ perspective-synth .04, babi-prose .01 Β· s8 definitions .25,
323
+ register-synth .25, simple-wiki .15, cosmo-young .12, wikipedia-en .10,
324
+ tinystories .09, recall-synth .03, siqa-narrative .01. Rails audited at
325
+ append time: MAX_EPOCHS 4.0, MAX_GENERATOR_SHARE 0.35, MIN_BALLAST 0.30
326
+ natural text (`curriculum.py:548-551,565-596`); holdouts enforced at
327
+ the generator β€” bAbI tasks 16/19, ProofWriter depth 5, rule-chain
328
+ depth 5, three-digit subtraction (`curriculum.py:26-31`). No chat
329
+ template in s0–s8.
330
+
331
+ **The anneal recipes** (`streams.py:68-77`): ANNEAL_MIX = fineweb-edu
332
+ .45, cosmopedia .20, tinystories .15, soda-dialogue .12,
333
+ beatrix-texture-sp .05 (the specials-native SYS/USER/MODEL/END
334
+ identity texture), recall-synth .03; ANNEAL_NOCHAT is the same list
335
+ minus beatrix-texture-sp, renormalised by MixStream β€” so the nochat
336
+ boundary is the exact pre-chat baseline on the same distribution.
337
+ MixStream draws one component per batch from a counter-seeded RNG
338
+ (`streams.py:375-404`): reproducible and resumable, but its state
339
+ carries no recipe fingerprint (fresh starts only across recipe
340
+ changes).
341
+
342
+ **The DOC plane on the diet.** The packer appends DOC after every
343
+ document (`streams.py:192-372`); the specials gauge reads `doc_bpb`
344
+ (bits at positions whose target is DOC) and `reset_bpb` (mean bits over
345
+ the 16 targets after each DOC) on the fixed val batches
346
+ (`instruments.py:310-352`). doc_bpb fell 12.83 β†’ 3.29 by 0.52B and
347
+ sits at 3.862 at the endpoint (control 5.422); reset_bpb 4.287 β†’ 2.130
348
+ (control 3.825). doc_count is 70 in every 2s and control report except the struck 2s
349
+ `curriculum_s8_step53792.json`, which reads doc_count 1612 (the synthetic
350
+ fallback stream) β€” the same 70 documents every time on the fineweb-edu gauge.
351
+
352
+ **The val gauge.** `_val()` builds 4 batches of (16, 4097) from
353
+ `build_stream(dataset, …, seed = 1337 + 9999, role="val")`
354
+ (`trainer.py:384-411`); for fineweb-edu the val region is the first
355
+ 2,048 unshuffled rows, which the training stream skips
356
+ (`streams.py:56,279-282`), and `build_stream` routes every
357
+ `curriculum-*` and `anneal-*` val request to that same fineweb-edu head
358
+ holdout β€” "gauge continuity" in the source (`streams.py:423-447`). So
359
+ from fineweb_main onward every row of both crafts is one gauge; the
360
+ canon's CB2/CB3 "own slice" wording is struck by erratum
361
+ (`canon/splat_aleph_battery.md:583`). **The s8 fallback caveat**: at the
362
+ terminal curriculum boundary `current_phase()` is None (everything done
363
+ or deferred) and pre-0.8.5 `_val` fell to the `synthetic` stream, so the
364
+ 2s `curriculum_s8_step53792` ledger is an incomparable artifact; 0.8.5
365
+ (@7d48328) makes `_val` fall back to the last DONE phase's dataset
366
+ (`trainer.py:397-401`), and the true end-of-curriculum ledger is
367
+ `session_cap_step53792`, taken at the same step on the restored gauge
368
+ (1.1911; banks +3.214, hubs +3.341, head +0.0149). The control, born
369
+ under 0.8.7, measured its s8 row on the repaired gauge; the s8 pair is
370
+ therefore not same-gauge and is struck from the paired count.
371
+
372
+ ## The life (`reports/v2/`, `resume/`, `runs/`)
373
+
374
+ **Sessions.** The 2s craft ran five sessions on the one RTX PRO 6000
375
+ (Blackwell, 96 GB; 4.54 s/step, ~58k tok/s), each its own event file:
376
+ steps 50–8,300 (host 4d130c0e7351), 8,350–24,850 (91b9f3518593),
377
+ 24,900–43,100 (e935248f0778), 43,150–53,750 (68304a2b8506),
378
+ 53,800–61,400 (c2dc3d694638). The breaks are the session caps at 8,320
379
+ (fineweb, 2.181B), 43,130 (inside s6, 11.306B) and 53,792 (the s8 boundary
380
+ itself, the resumed session's first action) β€” `session_cap_step*.json`
381
+ is written when a session hits MAX_HOURS mid-phase β€” and the revival
382
+ halt at 24,860. Sessions are boundary-exact: the driver calls
383
+ `train(max_hours, stop_at_boundary=True)`, which returns after each
384
+ boundary checkpoint with `_last_boundary` set, and never auto-resumes a
385
+ manual stop (`_interrupted`; `trainer.py:170-184,328-335`; the 0.8.1
386
+ driver contract, minted after run0 rolled through its boundaries and
387
+ swallowed a KeyboardInterrupt).
388
+
389
+ **Resume.** `_restore` loads `resume/latest.pt` β€” fp32 model, both
390
+ optimizer states, the stream position, torch and CUDA RNG, and the
391
+ manifest snapshot embedded in the payload, which wins over the hub's
392
+ `manifest.json` (`trainer.py:110-153`; `checkpoint.py:131-175`). A
393
+ missing payload over a manifest with steps > 0 refuses to start fresh
394
+ (`trainer.py:110-120`). Every phase boundary archives an immutable copy
395
+ `resume/boundary_<phase>_step<N>.pt` (`trainer.py:469`;
396
+ `checkpoint.py:134-154`). Resume is not row-exact: the datasets shuffle
397
+ buffer (10,000 rows, `streams.py:292-293`) is not part of the saved
398
+ state, so each resume refills it (`streams.py:12`, the honesty note).
399
+
400
+ **The revival at 24,860.** The born-null head buried itself inside the
401
+ first 2,000 steps (0.032Γ— chance liveness at the first stored
402
+ checkpoint; β€–sβ€– ~6e-4; proj cosines ~60Γ— below chance β€” learned
403
+ avoidance; `repos/alephllm.md:1564-1643` β€” journal readings from
404
+ checkpoint analysis: the craft's TB carries no `head/liveness_ratio`
405
+ scalar before 0.8.4; the control's TB reads 0.052Γ— @2,000, 0.014Γ—
406
+ @12,000, 0.010Γ— @24,000). The BOUNDARY-WRITE op
407
+ (`revival.py:44-104`) ran on the halted run: proj ← uncentered
408
+ whitening-PCA of the final-norm outputs (sv floor 1e-3 relative);
409
+ codebook ← fresh `normalize(randn)` with min-sep 45Β° (0 pushes needed);
410
+ W_s ← fp64 ridge on the base head's CE residual, Ξ± swept on the held-out
411
+ half and folded in. `revival_step24860.json`: K 256, D 256, seed 13,
412
+ sv0 3676.66, svD 15.90, Ξ± 8.0, β€–W_sβ€– 481.53, fit_positions 65,536,
413
+ base_bpb 1.27682 β†’ revived_bpb 1.23391 (Ξ” βˆ’0.04290 held-out),
414
+ liveness 0.01027Γ— β†’ 0.99642Γ—, version 0.8.2, commit d6e1854. The
415
+ pre-revival resume was archived, Muon momentum zeroed for the three head
416
+ tensors, and the revived payload promoted on Phil's explicit go
417
+ (`repos/alephllm.md:1669`); thereafter `head_addr_frozen` freezes proj
418
+ and the head codebook by `requires_grad` alone, so optimizer groups and
419
+ resume state are unchanged and Muon skips them (`trainer.py:75-80`;
420
+ `optim.py:42-43`). `checkpoints/step_00024860.safetensors` is the
421
+ pre-revival bf16 weight; `step_00024860_revived.safetensors` (fp32) is
422
+ the post-write weight; neither the revived file nor the resume archives
423
+ appear in `manifest.json`'s checkpoint index.
424
+
425
+ **The anneal.** Activated 2026-08-31T09:54Z after s8; nochat done
426
+ 14:42Z, chat done 19:31Z; the frame cost nothing in a bracketed phase
427
+ (probe mean .389 β†’ .389; val 1.1153 β†’ 1.1097).
428
+
429
+ **The 2s boundary ledgers** (`reports/v2/*.json`; bpb on the fineweb-edu
430
+ holdout; toggles = mechanism-off minus full):
431
+
432
+ | report | step | tokens | bpb_full | banks | hubs | head | doc / reset |
433
+ |---|---|---|---|---|---|---|---|
434
+ | warmup_wikitext | 1,145 | 0.300B | 2.36257 | +0.0365 | **+1.5042** | βˆ’2.3e-06 | 12.83 / 4.287 |
435
+ | session_cap | 8,320 | 2.181B | 1.20332 | +1.727 | +2.036 | βˆ’4.9e-07 | |
436
+ | fineweb_main | 20,219 | 5.300B | 1.11719 | +3.5527 | +2.2157 | βˆ’3.4e-07 | |
437
+ | curriculum_s0 | 22,890 | 6.000B | 1.2522 | | | ~0 (born null) | |
438
+ | curriculum_s1 | 25,561 | 6.700B | 1.2544 | | | **+0.0258** | |
439
+ | curriculum_s2 | 29,376 | 7.700B | 1.2177 | | | +0.0140 | |
440
+ | curriculum_s3 | 33,954 | 8.900B | 1.2318 | | | +0.0072 | |
441
+ | curriculum_s4 | 38,532 | 10.100B | 1.2356 | >3.0 | >3.0 | +0.0130 | |
442
+ | curriculum_s5 | 42,347 | 11.100B | 1.2504 | | | +0.0278 | |
443
+ | session_cap | 43,130 | 11.306B | 1.2614 | | | | |
444
+ | curriculum_s6 | 45,399 | 11.900B | 1.2683 | | | +0.0126 | |
445
+ | curriculum_s7 | 50,740 | 13.300B | 1.1883 | | | βˆ’0.0010 | |
446
+ | curriculum_s8 (STRUCK) | 53,792 | 14.100B | *1.0737* | *+3.396* | *+5.164* | | *0.843 / 1.889* |
447
+ | session_cap (true s8) | 53,792 | 14.100B | **1.19112** | +3.2144 | +3.3410 | +0.0149 | 5.052 / 2.469 |
448
+ | anneal_nochat | 57,607 | 15.100B | 1.1153 | | | +0.0101 | |
449
+ | anneal_mix | 61,422 | 16.101B | **1.10970** | +3.3115 | +3.3584 | +0.0123 | 3.862 / 2.130 |
450
+
451
+ Blank cells are not in the record consulted here; the head column is
452
+ complete. The warmup row is the first full-splat-under-AR ledger: with
453
+ all 20 hubs off the craft loses its entire attention (+1.50 at 0.3B),
454
+ which is what "load-bearing, not decorative" measures against the
455
+ mother's 3-hub form. `hidden_erank_floor` flags TRUE in every report from the 8,320 cap on (the
456
+ trainer's own TB census, on the first val batch, flags it from step 500)
457
+ (L19 pre-norm erank in single digits β€” the byte funnel; the post-norm
458
+ `nf(h)` erank reads 334/1024), `head_buried` FALSE from s1 on, den
459
+ floors 0.0 the entire run, zero loss spikes. Two "final val" readings
460
+ exist on the hub: the report ledger 1.10970 (the gauge of record) and
461
+ the manifest's checkpoint entry `val_bpb` 1.11403 at 61,422 β€” the
462
+ manifest column is the LAST PERIODIC EVAL's `bpb_full` (eval every
463
+ 2,000; `trainer.py:444-447`), so it lags every boundary (fineweb 1.1156
464
+ vs 1.1172; s8 1.1833 vs 1.0737/1.1911); the manifest minimum is 1.1140
465
+ @60,000. Cite the report.
466
+
467
+ ## The paired record (`mini-beatrix-2s-control/reports/v2/`)
468
+
469
+ **Derivation.** Every preset auto-generates a `-control` twin: the same
470
+ `AlephLMConfig` with `hub_layers = ()` (`presets.py:216-229`), so
471
+ `hub_K/hub_D/hub_const/hub_chunk` are carried but unused and every
472
+ block's attention is CausalSDPA. 0.8.7 (@b472815) gave twins their own
473
+ `TrainConfig` copy β€” the shared instance had let `head_addr_frozen=True`
474
+ leak into the control, a post-revival flag no birth recipe may carry β€”
475
+ and sets the 2s control to `head_addr_frozen = False` explicitly: the
476
+ birth recipe, born-null unfrozen head, which buries as the treatment's
477
+ did for its first 24,860 steps (`presets.py:221-235`). The control
478
+ inherits `governor="minsep"`; with no hub books it governs the head
479
+ codebook only (hits_cum 10 at warmup, fineweb and the 22,854 cap β€” the first
480
+ session β€” and 0 in every later session's report;
481
+ `crowd_check_extra_hits` 0). Same seed 1337: construction order embed β†’
482
+ blocks β†’ norm β†’ head after `torch.manual_seed` (`trainer.py:63`;
483
+ `alephlm.py:85-96`) makes the embedding draws identical between craft
484
+ and twin; the RNG streams diverge from block 0's attention onward
485
+ (a derivation from construction order, not a weight comparison).
486
+
487
+ **Protocol.** Declared at the control's warmup boundary, before the
488
+ outcome, while sdpa led (2.1056 vs 2.3626): the architecture verdict is
489
+ the paired boundary table across the full 16.1B matched budget; the
490
+ headline is endpoint val; secondaries are probe means at matched
491
+ boundaries and the doc/reset gauges; no verdict from any intermediate
492
+ lead in either direction (`repos/alephllm.md:1961-2002`; a control must
493
+ be able to fail). Completed unaltered through 13 paired rows. Budgets
494
+ are matched in tokens, not wall-clock: fused sdpa ran 71.9k tok/s on the
495
+ RTX PRO 6000 and 113k on the H100 against the eager scan's ~58k.
496
+
497
+ **The paired rows** (bpb_full, same step, same tokens, same gauge except
498
+ where struck; deficit = control βˆ’ craft):
499
+
500
+ | boundary | step | tokens | 2s | control | deficit | control bank toggle | control grad-norm median |
501
+ |---|---|---|---|---|---|---|---|
502
+ | warmup | 1,145 | 0.30B | 2.3626 | 2.1056 | βˆ’0.257 | βˆ’0.008 | |
503
+ | fineweb | 20,219 | 5.30B | 1.1172 | 1.1763 | +0.059 | 1.62 | 0.31 |
504
+ | s0 | 22,890 | 6.00B | 1.2522 | 1.5047 | +0.253 | 3.44 | 6.47 |
505
+ | s1 | 25,561 | 6.70B | 1.2544 | 1.3917 | +0.137 | 3.61 | 2.43 |
506
+ | s2 | 29,376 | 7.70B | 1.2177 | 1.3938 | +0.176 | 4.13 | 11.9 |
507
+ | s3 | 33,954 | 8.90B | 1.2318 | 1.5317 | +0.300 | 4.73 | 12.7 |
508
+ | s4 | 38,532 | 10.10B | 1.2356 | 1.5689 | +0.333 | 5.09 | 24.5 |
509
+ | s5 | 42,347 | 11.10B | 1.2504 | 2.1762 | **+0.926** | 4.36 | 126.4 |
510
+ | s6 | 45,399 | 11.90B | 1.2683 | 3.1940 | **+1.926** | 3.89 | 471 |
511
+ | s7 | 50,740 | 13.30B | 1.1883 | 3.1201 | +1.932 | 4.17 | 162 |
512
+ | s8 | 53,792 | 14.10B | *1.0737* | 3.0810 | STRUCK (gauges differ) | 5.48 | 145 |
513
+ | anneal_nochat | 57,607 | 15.10B | 1.1153 | 2.9270 | +1.812 | 5.64 | 167 |
514
+ | anneal_mix | 61,422 | 16.10B | **1.1097** | **2.8846** | **+1.775** | 6.11 | 215 |
515
+
516
+ Eleven same-gauge rows after the crossover, all in the craft's favour;
517
+ the endpoint gap is +1.775 at 16,101,408,768 tokens on both. The
518
+ control's own trajectory is the finding: 1.1763 at 5.3B β†’ 3.1940 at s6
519
+ β†’ 2.8846 at the end β€” a destabilised model at its endpoint, not a
520
+ trailing one. Its endpoint ledger: bank_off 8.99015 (toggle +6.1056),
521
+ head toggle βˆ’1.06e-05, `head_buried` TRUE at every row from fineweb on,
522
+ doc_bpb 5.422, reset_bpb 3.825; probes p0 .200 p1 .333 p2 .267 p3 .333
523
+ p4 .300 p5 .433 p6 .233 p7 .433 p8 .333 against the craft's .367 .367
524
+ .600 .200 .333 .567 .333 .333 .400 β€” ties at n = 30 (SE of a difference
525
+ .126). The control's bank dispatch entropy fell through the curriculum
526
+ (`collapse/dispatch_entropy_L5`: .94 at the s0 census, .75 @30,000, .48
527
+ @35,000, .22 at the s5 census; L0 .75 β†’ .47 @30,000; layer minimum .14
528
+ @42,500), ahead of the >30 grad-norm-median onset at 38,000.
529
+
530
+ **The destabilisation facts** (from the hub tfevents, `train/grad_norm`
531
+ = pre-clip): the control first crossed the clip at step 17,600 (4.61B,
532
+ inside fineweb, on the RTX PRO 6000, while still leading); 813 of 824
533
+ logged curriculum+anneal steps clipped (98.7%); 38 steps above 1,000;
534
+ max 41,070 @44,600; the > 30 onset at bin 38,000 (late s4) and > 100 at
535
+ 40,500 (mid-s5) inside one event file. The 2s craft: 0 of 1,225
536
+ logged steps from step 200 on above the clip (the only reading above 1.0 is
537
+ step 50, 1.063, inside the 200-step LR warmup); max 0.760 @1,300 after the
538
+ warmup boundary, and 0.619 @26,750 (s2) over the 824 curriculum+anneal steps. The spike
539
+ detector and the non-finite guard never fired on either craft. Under
540
+ Muon the clip braked nothing in the Muon group (qkv/o, the position
541
+ table, the head); the escalating norm is a sharpness signal, not a
542
+ throttled step.
543
+
544
+ **The grade and its caveats.** The instance holds: the all-sdpa twin is
545
+ a destabilised control under the splat-tuned recipe, one craft per arm
546
+ (S-1802 INSTANCE). The general claim β€” softmax attention is inadequate
547
+ for this model's attention requirement β€” is a CANDIDATE (S-1803) behind
548
+ confounds stated as measurements: (a) the recipe was tuned on the splat
549
+ craft and carries none of the field's softmax stabilisers (QK-norm,
550
+ softcap, z-loss, weight decay, decay), and the signature β€” clip-engaged
551
+ smooth degradation, entropy and rank collapse, no NaN, onset on plain
552
+ prose while leading β€” is the documented softmax logit-growth mode;
553
+ (b) n = 1, no bar, +20.3M parameters in the losing arm; (c) shape: the
554
+ mother null β€” mini-beatrix-1 (13 sdpa + 3 hubs, d768) under the
555
+ identical TrainConfig (git cc89a87..b472815; the manifests record no
556
+ train_config) through the same nine stages at the same budgets ran a
557
+ post-warmup grad-norm max of 0.883, 0 of 1,750 logged steps clipped
558
+ (`canon/splat_aleph_battery.md:955`; canon-cited, not re-verified against
559
+ the mini-beatrix-1 event files for this companion β€” her 88,508 steps /
560
+ 26,102,071,296 tokens / 70.45 h do verify against
561
+ `mini-beatrix-1/manifest.json`) β€” with the nuance that v1 entered
562
+ s0 at 17.3B tokens after 15B prose and a 2B anneal, the control at 5.3B;
563
+ (d) the aleph-addressed bank sits in both crafts, its dispatch collapse
564
+ precedes the escalation, and no sdpa + dense-FFN arm exists at any
565
+ scale; (e) the 4096Γ—1024 position table on WD-free Muon at 2e-2 forever;
566
+ (f) house Muon is not reference Muon; (g) host and row-order
567
+ asymmetries. One gauge (val bpb; probes are ties); the s8 pair struck;
568
+ three hosts. dist-e002 (vision, two seeds) remains the only Law-E
569
+ splat-vs-sdpa endpoint, and it is a tie.
570
+
571
+ **Provenance of the three hosts.** RTX PRO 6000 from birth to the 23.5 h
572
+ cap at 22,854 (event file 50–22,850); a Colab H100 80 GB from 22,854
573
+ (~2.3–2.4 s/step) through s0–s8 (22,900–53,750), 36 steps before the s0
574
+ boundary, with the shuffle buffers refilled; a session 53,800–56,000
575
+ follows the 53,792 cap in the H100 session's own host and process (event
576
+ file `…0a706d592fec.620.1`, 20:01–21:35Z, ~2.3 s/step, 114k tok/s); a third
577
+ host (2b1783ef9e94) opened 2026-09-03T01:41Z from the step-56,000
578
+ checkpoint at ~3.6 s/step and ran both anneal rows (56,050–61,400; two
579
+ 88-byte event files are aborted starts) β€” a replay of up to 999 steps at
580
+ that restart is a hedge on the 1,000-step TB-upload cadence, not a hub
581
+ reading (the H100 process's last event is 21:35:20Z, the step-56,000
582
+ checkpoint landed 21:35:33Z and its manifest 21:36:06Z, the third host
583
+ opened 01:41:15Z; the hub can neither confirm nor refute a replay), and
584
+ it left no cap report.
585
+ The canon's "on the H100" lines for boundaries 11–12 are wrong by
586
+ erratum (`canon/splat_aleph_battery.md:1008`). The escalation began
587
+ pre-resume on the RTX PRO 6000; a 36-step data-order change cannot make
588
+ a 0.25 bpb gap on a fixed eval.
589
+
590
+ ## The boundary report (`reports/v2/`) β€” schema
591
+
592
+ Written by the notebook's `boundary_report(run, tag)` cell to
593
+ `reports/v2/{tag}_step{step}.json`, tag = the phase that just finished
594
+ or `session_cap`:
595
+
596
+ - `tag`, `step`, `tokens` β€” `tokens` is `manifest.tokens_seen` (new
597
+ in the v2 record).
598
+ - `probes` β€” `{p0_grounding, p1_self_other, p2_difference, p3_logic,
599
+ p4_arithmetic, p5_causal, p6_learning, p7_composite,
600
+ p8_articulation}`, each `{acc, families{…}, n: 30}`; byte-NLL argmin
601
+ over the options, fp32, no sampling (`probes.py:191-221`); holdout
602
+ families p3:d5 and p4:sub3 gauged from time zero (`probes.py:171`).
603
+ - `census_flags` β€” `hidden_erank_floor` (any block-output effective
604
+ rank < 0.05Β·d = 51.2), `anchor_merge` (any pair |cos| > 0.99 in a
605
+ bank/hub/head book), `dispatch_collapse` (bank dispatch entropy
606
+ fraction < 0.10), `den_floor` (hub den floor_frac > 1e-3),
607
+ `head_buried` (liveness < 0.1; added at 0.8.2 β€” absent from the 2s
608
+ warmup_step1145, session_cap_step8320, fineweb_main_step20219 and
609
+ curriculum_s0_step22890 reports, present from s1; present in every
610
+ control report),
611
+ `codebook_erank_floor` (bank codebook erank < 1.5)
612
+ (`instruments.py:150-178`). The census sample is `(2, min(1024,
613
+ ctx))` random bytes in 0..254 (`torch.randint(0, 255, …)`, so DOC never appears in the census sample) in the notebook writer; the trainer's own census
614
+ uses the first val batch (`trainer.py:371-374`).
615
+ - `ledger` β€” `bpb_full`, `bpb_bank_off`, `bpb_head_aleph_off`,
616
+ `bpb_hub_off` (only if any block is a hub β€” the control's ledger has
617
+ five keys), and `toggle_<x> = bpb_<x> βˆ’ bpb_full`; mean CE over the
618
+ fixed val batches / ln 2 (`instruments.py:187-212`). No `w_s_norm`
619
+ field (the mb1 schema had one).
620
+ - `special` β€” `{doc_count, doc_bpb, reset_bpb}` (`instruments.py:
621
+ 310-352`); None on BPE crafts.
622
+ - `governor_hits_cum` β€” the session's cumulative pair-pushes; the
623
+ counter is a Trainer attribute defaulting to 0 (`trainer.py:266`),
624
+ not persisted in the manifest or the resume payload, so it resets
625
+ every session (103,951 @1,145; 1,304,497 @20,219; 70,747 at s1 after
626
+ 1,604,488 at s0; 0 at session_cap_53792; 776,450 @61,422).
627
+ - `crowd_check_extra_hits` β€” pushes from an on-demand `govern_model` at
628
+ report time (415 @20,219; 709 @61,422; 0 on the control).
629
+
630
+ Other report kinds in the same directory: `revival_step24860.json` (the
631
+ op record above, no probes/ledger β€” chart code must skip it) and
632
+ `cultivation_{A_born_null,B_revival_birth,C_matured_transplant,
633
+ D_live_address_only}.json` β€” the C7 head-cultivation screen, 1,200
634
+ steps at seed 1337 with gauges at 300/600/900/1200 `{toggle, w_s_norm,
635
+ head_s_norm, head_s_chance, head_liveness_ratio, head_buried}` and,
636
+ for B and D, a `revival` dict (Ξ± 16.0, fit 16,384, base 8.336 β†’ 5.389).
637
+ Endpoint toggles: A βˆ’5.3e-06 (buried, liveness 0.045); B +3.496
638
+ (liveness 1.125); C +0.0922 (β€–W_sβ€– 520.5); D βˆ’7.1e-05 (liveness 1.037,
639
+ not buried, no function) β€” a live address alone does not carry
640
+ function; a revived head at birth does.
641
+
642
+ **The manifest** (`manifest.py:16-110`): `preset`, `model_config` (the
643
+ full `AlephLMConfig`), `created_utc`, `updated_utc`, `steps`,
644
+ `tokens_seen`, `wall_hours`, `phases[{name, dataset, planned_tokens,
645
+ tokens_done, status ∈ planned|active|done|deferred}]`,
646
+ `checkpoints[{step, tokens, kind ∈ safetensors|resume|fp8, path,
647
+ val_bpb, utc}]`, `data_state{dataset, rows_consumed, epoch}`,
648
+ `notes[{utc, msg}]`. 2s: created 2026-08-27T00:35:31Z, updated
649
+ 08-31T19:37:50Z, 142 entries (59 safetensors / 24 fp8 / 59 resume β†’ 46
650
+ distinct bf16 paths). Control: created 08-31T21:26:32Z, updated
651
+ 09-03T07:16:38Z, 136 entries β†’ 44 bf16. Neither records a train_config.
652
+
653
+ **Checkpoint kinds** (`checkpoint.py:102-155,187-195`): bf16 safetensors
654
+ every 2,000 steps and at every boundary, every state-dict tensor
655
+ including the `addr.home` buffers (so the file's numel, 237,908,285,
656
+ exceeds the parameter count by 782,336); fp8-e4m3 every 5th checkpoint
657
+ or final, 2-D+ tensors with numel > 4,096 stored with a per-tensor fp32
658
+ scale `amax/448` under `<key>.scale`, small params kept bf16, metadata
659
+ "inference/testing only" β€” never train from these; `resume/latest.pt`
660
+ fp32 with the manifest embedded, overwritten at every checkpoint and
661
+ archived at boundaries. Hub uploads retry three times and never crash
662
+ training; downloads distinguish genuine absence from network trouble
663
+ and raise on the latter (`checkpoint.py:29-100`).
664
+
665
+ **TensorBoard tags** (`trainer.py:283-296`; `instruments.py:215-260`;
666
+ 314 tags on the craft, 212 on the control): from the loop
667
+ `train/{loss,bpb,grad_norm,lr_scale,tokens,tokens_per_sec}`,
668
+ `collapse/loss_spike`, `governor/hits_cum`, `sys/vram_gb`; from the
669
+ census per layer i `erank/hidden_L{i}`, `erank/hub_consumed_L{i}`,
670
+ `erank/hub_codebook_L{i}`, `erank/bank_codebook_L{i}`,
671
+ `cv/bank_load_L{i}`, `cv/hub_usage_L{i}`, `drift/bank_L{i}`,
672
+ `drift/hub_L{i}`, `sign/bank_frac_neg_L{i}`,
673
+ `collapse/dispatch_entropy_L{i}`, `collapse/den_floor_L{i}`,
674
+ `gates/L{i}_e{k}`; head `head/{gamma,w_s_norm,s_norm,liveness_ratio,
675
+ consumed_erank}`, `erank/head_codebook`; `collapse/flag_{…}` for the six
676
+ flags; `toggles/{bpb_full,bpb_bank_off,bpb_head_aleph_off,bpb_hub_off,
677
+ toggle_bank_off,toggle_head_aleph_off,toggle_hub_off}`; from the full
678
+ eval `canary/{recall_acc,wrongpair_frac,noretrieval_frac}` and
679
+ `special/{doc_count,doc_bpb,reset_bpb}`. The control lacks every hub
680
+ family; the archived run0 file lacks `head/liveness_ratio`,
681
+ `head/s_norm`, `special/*` and `flag_head_buried` (pre-0.8.0 specials;
682
+ the `head_buried` flag arrived at 0.8.2 @d6e1854 and the `head/s_norm` +
683
+ `head/liveness_ratio` TB scalars at 0.8.4 @a690ea4).
684
+
685
+ **`archive/run0-nospecials/`.** The pre-specials attempt, archived whole
686
+ 2026-08-27T00:16Z: its manifest (created 08-26T20:50:18Z, steps 2,537,
687
+ tokens 665,059,328, wall 3.22 h, warmup done at 1,145, fineweb_main in
688
+ progress), bf16 checkpoints 1,145/2,000/2,253/2,537, fp8 2,253/2,537,
689
+ `resume/{latest,boundary_warmup_wikitext_step1145}.pt`, one event
690
+ file, and a report at the doubled path
691
+ `archive/run0-nospecials/mini-beatrix-2s/reports/v2/warmup_wikitext_step2253.json`
692
+ β€” a late-tagged fineweb report written on interrupt (stale tag, doubled
693
+ path: the 0.8.1 driver-contract bug) with probes, five census flags and
694
+ governor fields but no `ledger`/`special` block. Code-wins note: the
695
+ manual's RIDER 9 says run0 "halted at step 2000"; the archived manifest
696
+ says 2,537 (train() swallowed the interrupt and the driver auto-resumed
697
+ ~19 minutes; `repos/alephllm.md:1408`). The hub wins: 2,537.
698
+
699
+ ## The AutoModel (AbstractPhil/mini-beatrix-2s)
700
+
701
+ Thirteen files: `README.md`, `config.json`, `modeling_minibeatrix.py`,
702
+ the vendored `address.py`, `alephlm.py`, `attention.py`, `bank.py`,
703
+ `embedding.py`, `governor.py`, `head.py`, `presets.py`,
704
+ `model.safetensors` (475,880,498 B; 593 bf16 tensors, numel
705
+ 237,908,285, every key prefixed `model.`, metadata `{tokens: 16.101B,
706
+ step: 61422, format: pt}`), `.gitattributes`. No tokenizer files: input
707
+ ids are raw UTF-8 byte values. The vendored files are byte-identical to
708
+ `geolip/alephllm/model/*.py` and `presets.py` at 0.8.7 except one
709
+ import line (`alephlm.py:30`, `..presets` β†’ `.presets`). `config.json`
710
+ is the craft's `model_config` plus `model_type "mini-beatrix"`,
711
+ `auto_map` to `MiniBeatrixConfig`/`MiniBeatrixForCausalLM`,
712
+ `torch_dtype "float32"` (the file is bf16; the loader upcasts on
713
+ request), `use_cache false`. The wrapper (`modeling_minibeatrix.py:
714
+ 21-99`) holds `self.model = AlephLM(cfg)`, forces `hub_ckpt = 0`, makes
715
+ `_init_weights` a no-op ("AlephLM initializes itself; loader must not
716
+ re-init"), truncates to the last `context` positions, returns
717
+ `CausalLMOutputWithPast(loss, logits)` with HF label semantics, and
718
+ carries no KV cache β€” HF `generate()` recomputes the prefix each step;
719
+ the inner `AlephLM.generate(idx, max_new, temperature, top_p,
720
+ use_cache=True)` uses the constant-size hub state.
721
+
722
+ **The parity gate.** The ship gate β€” strict load, logits parity 0.0
723
+ against the geolip code path, `generate()` ok β€” is journal-recorded
724
+ (`repos/alephllm.md:1840-1846`; `history/timeline.md` 2026-08-31;
725
+ `MIRROR.md` step 2 in the AlephLLM repo), not hub-shipped: no parity
726
+ script exists on either hub repo or in the package. Re-verified on CPU
727
+ by the drafting workflow (torch 2.11.0, transformers 5.13.0, no geolip
728
+ installed; not re-run in the hub-side pass, which found the hub
729
+ consistent with these readings β€” TB `head/w_s_norm` 515.13 at step
730
+ 61,000, 593 bf16 tensors, numel 237,908,285): 593 file keys == 593 model keys, parameter count
731
+ 237,125,949, β€–W_sβ€– 515.918, Ξ³ 1.0; the inner cached decode equals the
732
+ parallel path (`torch.equal` True); logits parity against geolip was
733
+ not recomputed. Two notes for the reader: the model card's lineage line
734
+ says alephllm 0.8.6 while the vendored files are the 0.8.7 mirror
735
+ (commit 90a06b75, 2026-08-31 21:11 β€” the `presets.py` carries the 0.8.7
736
+ twin-isolation block); and the card's "17 reports" counts the 16
737
+ boundary/session reports plus the revival op β€” the prefix holds 21
738
+ JSONs with the four cultivation arms.
739
+
740
+ **Load and evaluate** (the steps below were executed by the drafting
741
+ workflow; the numbers are from that run and were not re-run in the
742
+ hub-side pass):
743
+
744
+ ```python
745
+ import math, torch
746
+ from transformers import AutoModelForCausalLM
747
+ m = AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-2s",
748
+ trust_remote_code=True).eval()
749
+ ids = torch.tensor([list(text.encode("utf-8"))]) # bytes in
750
+ bpb = m(input_ids=ids, labels=ids).loss.item() / math.log(2)
751
+ # toggle ledger on the inner model (positive delta = mechanism carries function)
752
+ core = m.model
753
+ full = core(ids, labels=ids).loss / math.log(2)
754
+ bank_off = core(ids, labels=ids, disable_bank=True).loss / math.log(2)
755
+ hub_off = core(ids, labels=ids, disable_hub=True).loss / math.log(2)
756
+ head_off = core(ids, labels=ids, disable_head_aleph=True).loss / math.log(2)
757
+ # generation: HF path (no cache) or the inner cached decode
758
+ out = core.generate(ids, max_new=40, temperature=0.0, use_cache=True)
759
+ print(bytes(out[0].tolist()).decode("utf-8", errors="replace"))
760
+ ```
761
+
762
+ On a 227-byte English snippet this read 0.9446 bpb full, +3.872 bank
763
+ off, +3.925 hub off, +0.0156 head off; greedy from "The history of
764
+ astronomy begins with the" continued " study of the sun and the
765
+ planets. The s". Chat: wrap raw bytes in 0xFB SYS / 0xFE USER / 0xFD
766
+ MODEL / 0xFC END (`special_tokens.py:112-136`). Training checkpoints
767
+ load into the same wrapper after prefixing every key with `model.`, or
768
+ into a bare `AlephLM(cfg.to_aleph())`; fp8 files must be rehydrated
769
+ first (`checkpoint.py:187-195`). What the AutoModel cannot reproduce is
770
+ the gauge of record: the fineweb-edu holdout needs
771
+ `geolip.alephllm.data.streams.build_stream(dataset, tok, 4096, 16, seed
772
+ = 1337 + 9999, role="val")` and 262,144 tokens (4 batches of 16 Γ— 4096),
773
+ then `toggle_ledger` and `special_token_gauge` over those fixed batches
774
+ (`trainer.py:384-411`; `instruments.py:187-212,310-352`).
775
+
776
+ ## The process that made 2s different (dated)
777
+
778
+ **The mother, measured (2026-08-12 β†’ 08-27).** mini-beatrix-1 was built
779
+ with 13 sdpa blocks and three hubs at layers 4/9/14, each a single
780
+ 512-anchor book in D = 32 (`git show cc89a87:presets.py`), full-width
781
+ banks holding 67% of the mass, and a 512 @ 32 head. Two post-mortems set
782
+ the 2s agenda: the 08-25 hub census β€” 16Γ— the K ≀ 2D supply, 333/646/181
783
+ duplicate anchor pairs at |cos| ~1.0, consumed erank 2.2/3.7/10.8 β€” and
784
+ the 08-27 autopsy of the shipped weights, W_s exactly 0.0 after 8.4B:
785
+ the "head election" lore retracted, the head never carried function
786
+ (`repos/alephllm.md:1586`; `history/plans/2026-08-26_mini_beatrix_v2_shape.md`).
787
+ A third: the v1 anneal (2B, chat template inside, before the school)
788
+ baked the chat frame into the core irreversibly β€” the chat-in-anneal
789
+ law (2026-08-15) that later dictated the bracketed two-phase anneal.
790
+
791
+ **Why every block (08-23 β†’ 08-26).** Phil's directive: "the entire model
792
+ to be splat attention". The record's support was dist-e002 β€” cured
793
+ splat in all 12 ViT-Ti blocks ties sdpa at matched params and matched
794
+ wall-clock on two seeds (R@1 .0545 vs .0562), late placement edging
795
+ (R@5 +.0128, outside the bars); the untested bet, flagged in the plan,
796
+ was that full-splat CAUSAL had never trained. The payoff row was the
797
+ warmup ledger (+1.50 with the hubs off at 0.3B) and the endpoint
798
+ (+3.36).
799
+
800
+ **The shape (08-24/25 β†’ 08-26).** ROUND 5e's shape ladder: 8 Γ— 256 lost
801
+ 4 of 4 paired against 12 Γ— 128, the wide model seed-fragile, its
802
+ crowding tail (79% of layer-1 heads over the 25Β° floor on one seed)
803
+ naming the chain width β†’ crowding β†’ positional ranking flips β†’ deficit;
804
+ hence SUPPLY K ≀ 2D (`STATUTES_INDEX.md:53`) and depth over width. B2's
805
+ product code (independent books compose, .859 β†’ .955 monotone in
806
+ members) gave the multi-book form; the rotary Γ— anchor flip analysis
807
+ gave governability D β‰₯ 128 (Ξ³* 32Β° at d512, 45Β° at 256, unreachable at
808
+ 64). The arithmetic lands on 4 Γ— 64 @ 128 (0.5Γ— supply, 4Γ— headroom
809
+ against v1's 16Γ—), L20, d1024, ctx 4096 (splat's wall-clock crossover
810
+ ~2k; below 4k a full-splat trunk pays attention's price without the
811
+ O(L) payoff), banks narrowed to ff 1024, head 256 @ 256. The 849M
812
+ sibling (L32, ctx 8192, 16 Γ— 256 @ 256) was designed the same session
813
+ (`presets.py:161-183`).
814
+
815
+ **The governor from birth (08-24 β†’ 08-26).** Phil's proposal β€”
816
+ "a middleman to control a min/max space that drift can occur to
817
+ guarantee there's always enough space between the anchor points" β€” was
818
+ confirmed in the math: for a crowded pair the static logit gap is
819
+ quadratic in separation while the positional oscillation is linear, so
820
+ small separation flips weight rankings along the sequence; separating
821
+ an already-crowded trained book leaves its flip pattern bit-identical,
822
+ so min-sep is preventive, not curative β€” run it from birth. Rung 1 ran
823
+ bit-identical to the ungoverned run (raw books never crowd; the
824
+ crowder was the frame W_a); gov25m fired on the fragile seed only,
825
+ harmlessly. Shipped as a post-step projection, 45Β°, every 8 steps
826
+ (`governor.py`, `presets.py:99-105`). In-mission it ran hot β€” ~120
827
+ pushes/step at step 1,000, ~84–105 sustained through fineweb, never
828
+ decaying β€” with every function gauge healthy; the ΞΈ ladder 35/40/45 for
829
+ K = 64 @ D128 is an open rung, and the preset comment's "45 at D=256"
830
+ does not itself justify 45Β° at D = 128.
831
+
832
+ **The kernel campaign and the pivot (08-26, one day).** 0.7.0 at 10:41
833
+ (multi-constellation hub, supply guard, governor wired in, the 849M
834
+ rescale, 2s as the screen craft). The Blackwell preflight OOMed
835
+ (`hub_ckpt`), then launched at 161.88 s/step; the per-op profiler
836
+ printed `qc torch.float32` under autocast β€” 0.7.2 at 12:27 (the einsum
837
+ promote trap; 4090 hub fwd+bwd 229.7 β†’ 178.8 ms). The dtype fix bought
838
+ ~6%; the binding constraint was kernel shape β€” 0.7.3 at 13:14 (one
839
+ stacked GEMM, one fused softmax, prefix as a triangular GEMM; 65.3 ms,
840
+ 3.5Γ—; peak 7.4 β†’ 1.9 GB; Blackwell 149 β†’ 69.7 s/step). Compile refuted
841
+ by measurement (bf16-backward NaN, and the pre-NaN projection 67 vs
842
+ 69.7 s eager) β€” eager-only on Blackwell. Economics: 17.3B at 8k tok/s
843
+ is 24–26 days on the one card. Phil: "we can't train the large one
844
+ currently. Not until we work out a series of logistics problems." 0.7.5
845
+ at 13:43 (@73218bd): 2s is the mission, `hub_ckpt` 2 β†’ 0; launched at
846
+ 4.54 s/step, 58k tok/s. "Not a bad start. We'll train this one." The
847
+ 849M is shelved, not retired (its value-bottleneck relaunch lever
848
+ quantified at ~46 s/step).
849
+
850
+ **The format restart (08-26 16:43).** Phil: "We'll want a practical
851
+ array of them for usage, nothing small. We'll halt this one and start
852
+ over with the correct formatting." 0.8.0: the thirteen unreachable ids,
853
+ the exhaustive proof at every launch, DOC from birth, the frame only in
854
+ anneal, the canary collision caught; run0 archived whole, run1 fresh on
855
+ the clean prefix (manifest 08-27T00:35Z). DOC cost nothing β€” run1 train
856
+ bpb 1.49 @1,100 against run0's 1.494 @1,000 β€” and doc_bpb fell 12.83 β†’
857
+ 3.29 by 0.52B. 0.8.1 the same day: the driver contract that made every
858
+ later boundary row a real boundary.
859
+
860
+ **The head saga (08-13 β†’ 08-29 β†’ 08-31).** Born weight-zero by law
861
+ (gate-zero deadlock, `head.py:8-11`); buried inside 2,000 steps on run1
862
+ (0.032Γ— chance at the first stored checkpoint, 0.014Γ— @12k β€” journal
863
+ readings from checkpoint analysis, not TB scalars; learned
864
+ avoidance β€” proj rotated into codebook-orthogonality while W_s grew to
865
+ norm 31 multiplying ~0); LS-ignite alone refuted (0.0006 bpb); the
866
+ governor exonerated (the ungoverned v1 buried deeper; the 45Β° floor is
867
+ free at birth). Candidate law at 2/2 crafts, 3/3 with the control: a
868
+ born-null aleph head racing a strong same-target linear base
869
+ self-buries within ~2k steps; the bank's self-start does not transfer
870
+ across competitor types. Phil ruled activation; the BOUNDARY-WRITE law
871
+ was minted (`revival.py:26`; `STATUTES_INDEX.md:67`) β€” a third
872
+ intervention category outside both the governor statute and the
873
+ no-balance-machinery law, permitted only at run boundaries or halted
874
+ resume points, always logged β€” and the deterministic op executed at
875
+ 24,860. The first nonzero head toggle in lineage history landed at s1
876
+ (+0.0258, after 6.5B tokens of exactly 0.000); the settled read is a
877
+ durable, small, diet-sensitive niche in the band βˆ’.001..+.028,
878
+ band-exits upward on the binding-adjacent diets (s5 causal +.0278),
879
+ touching zero at s7 as bpb_full fell 1.268 β†’ 1.188.
880
+
881
+ **The recipe that never changed (08-12 β†’ 09-04).** Muon on the trunk
882
+ because the 08-10 validity battery on the AR bed read muon .8081 >
883
+ sgd_mom .7995 > adam .7157 > lion .5638, with the address ~20Γ— more
884
+ optimizer-fragile than sdpa (cross-optimizer spread .79 vs .037) and
885
+ schedules no-op-to-harmful (flat .714 > warmcos .596) β€” so the flat-LR
886
+ law; pure Adam on the rest by the 09-01 optimizer statute (Adam the
887
+ experimental baseline, Muon the trunk, AdamW never;
888
+ `STATUTES_INDEX.md:76`). `presets.py:80-84` is the same text at
889
+ cc89a87 (where it sits at `presets.py:65-69`) and at HEAD. The same recipe is the audit's confound (a).
890
+
891
+ **The diet (08-15 β†’ 08-31).** The nine stages and budgets are v1's
892
+ (0.6.0), placed before the anneal rather than after it (Phil,
893
+ 2026-08-30: "Let her really build up that internal knowledge first";
894
+ `history/plans/2026-08-26_mini_beatrix_2s_mission.md`), fineweb_extended
895
+ deferred; the two-phase anneal (Phil, 08-31: "enable the anneal mix
896
+ without chat and the anneal mix with chat, we'll run without first,
897
+ then just train chat on after"; `streams.py:63-77`), 0.8.6 at 02:48.
898
+ The s8 terminal-gauge hole appeared at 04:57Z and 0.8.5 had already
899
+ closed it in code (08-30 22:03); the true ledger was re-measured at
900
+ 09:53Z. Mission complete 19:31Z: 61,422 steps, 16.101B tokens, 78.33 h,
901
+ zero spikes; the HF outage at the final push was recovered by a Drive
902
+ rescue and retry.
903
+
904
+ **The twin (08-31 β†’ 09-04).** 0.8.7 at 14:10 (twin isolation, birth
905
+ recipe); the control born 21:26Z; the protocol declared at its warmup
906
+ boundary while sdpa led; the crossover at fineweb 5.3B and every row
907
+ after; the rank story as the mechanism claim (at step 4,000 the control
908
+ collapses early hidden erank to 5–9 of 1024 at L2–L4 (5.5 / 5.4 / 9.3)
909
+ while the craft reads 234 / 225 / 250 there and 150–396 across L0–L15
910
+ (`erank/hidden_L{i}`, TB census); screen scale reproduces it; rank β‰  function β€” the
911
+ control led val at 4k while running rank-5 layers); the H100 resume at
912
+ 22,854; the grad-norm escalation rider at s5; Phil's claim held on
913
+ 09-02 ("SDPA isn't an adequate representative of this model's
914
+ attention requirement. We can say that without question now."); the
915
+ third host from 56,000; the endpoint 09-03; the 12-agent audit on 09-04
916
+ (S-1802 INSTANCE, S-1803 CANDIDATE, the s8 strike, the mother null,
917
+ Muon clip-blind, the falsifier filed).
918
+
919
+ ## Gauges, defined once
920
+
921
+ - **val bpb**: mean cross-entropy in bits/byte over 262,144 tokens
922
+ (4 batches of 16 Γ— 4096) of the fineweb-edu reserved head slice β€”
923
+ the first 2,048 rows, fixed, unshuffled, seed 1337 + 9999 β€” for every
924
+ phase from fineweb_main onward, both crafts (`trainer.py:384-411`;
925
+ `streams.py:279-282,423-447`; gauge continuity law). The warmup row
926
+ vals on wikitext-103's validation split. Pre- and post-0.8.0 gauges
927
+ differ for v1-era crafts (DOC rides the val stream).
928
+ - **toggle ledger**: bpb with one mechanism disabled minus full bpb β€”
929
+ the causal contribution of banks / hubs / head aleph
930
+ (`instruments.py:187-212`). Positive = the mechanism carries function.
931
+ - **specials gauge**: `doc_bpb` and `reset_bpb` in bits, read against
932
+ the same val batches (`instruments.py:310-352`); `doc_count` 70 on
933
+ every 2s-era report on the fineweb-edu gauge (the struck 2s s8 file reads 1612).
934
+ - **head liveness**: mean β€–signed readβ€– of real hidden states over the
935
+ same book's mean on 512 random unit vectors β€” self-calibrating;
936
+ `head_buried` = ratio < 0.1 (`instruments.py:144,176,356-382`).
937
+ Both buried crafts sat at 0.003×–0.03Γ— chance.
938
+ - **probe accuracy**: byte-NLL argmin over the item's options, nine
939
+ suites Γ— 30 items, holdout families d5 and sub3 gauged from time zero
940
+ (`probes.py:171-235`). At n = 30 a between-craft difference has SE
941
+ .126: the paired probe calls are ties.
942
+ - **train/grad_norm**: the pre-clip global norm returned by
943
+ `clip_grad_norm_` (`trainer.py:244,287`), logged every 50 steps; under
944
+ Muon it measures sharpness, not the applied step.
945
+ - **governor hits**: cumulative pair-pushes per session
946
+ (`trainer.py:266`); a session-scoped counter, comparable within one
947
+ event file only.
948
+
949
+ ## Known caveats and open items
950
+
951
+ - The autocast einsum-promote trap voids every pre-0.7.2 splat speed
952
+ verdict (fp32 scans under "bf16"); accuracy verdicts stand. Any
953
+ autocast performance reading over einsum code is invalid until the
954
+ dtypes are printed (`attention.py:167`).
955
+ - Muon is clip-blind (`optim.py:24`), and Adam's per-coordinate
956
+ normalisation makes it nearly so: `grad_clip 1.0` shrank the applied
957
+ step of neither group on the control, only the weighting of steps
958
+ inside each optimizer's buffers; the recipe carries no softmax
959
+ stabiliser.
960
+ - The 2s `curriculum_s8_step53792` ledger is on the synthetic fallback
961
+ and is struck; the same-gauge pair is owed by a zero-training
962
+ re-evaluation of `resume/boundary_curriculum_s8_step53792.pt` on the
963
+ 0.8.7 gauge (T0b in `history/plans/2026-09-04_sdpa_falsifier_screen.md`).
964
+ Hub-vs-journal note: the file on the hub reads doc_count 1612,
965
+ bpb_hub_off 6.2378 (toggle_hub_off 5.164), doc_bpb 0.843 β€” agreeing
966
+ with the trainer comment and the manual (doc_count 1612, hub_off 5.16 =
967
+ the toggle; `trainer.py:397-399`); the file carries no dataset field,
968
+ so the strike rests on the same-step cap discrepancy (doc_count 70,
969
+ bpb_full 1.1911 in `session_cap_step53792.json`) and on the 1612-document
970
+ count itself. The hub file's numbers win for what the file says.
971
+ - The control's third host from 56,000: a replay of up to 999 steps is a
972
+ TB-cadence hedge, not a hub reading; no cap report; the 53,800–56,000 segment ran in the H100
973
+ session's own process (event file `0a706d592fec.620.1`); canon CB11/12 "on the H100" wrong by erratum.
974
+ - Every resume refills the shuffle buffer (not row-exact); the control
975
+ resumed three times, the craft four; MixStream state carries no
976
+ recipe fingerprint.
977
+ - The 2s preset at HEAD carries `head_addr_frozen = True`
978
+ (`presets.py:199-201`), the post-revival state: a fresh birth of the
979
+ treatment preset under 0.8.7 would freeze a random proj and codebook
980
+ from step 0, which is not what the shipped run did (born-null unfrozen
981
+ under 0.8.1 for 24,860 steps, revived, then frozen). The control's
982
+ override (`presets.py:235`) is the birth recipe.
983
+ - Stale text, code wins: `presets.py:18` "~233M" (built 237,125,949);
984
+ `presets.py:14,166` "~873.7M" for mini-beatrix-2 (built 849,023,425);
985
+ the training README's ladder row "mini-beatrix-2 | 1024/20/2048 |
986
+ 249.1M" is the retired v1 shape; the hub driver `beatrix_v2_colab.ipynb`
987
+ is headed "mini-beatrix-2 β€” the governed full-splat mission" and names
988
+ `mini-beatrix-2-control` while its CRAFT is `mini-beatrix-2s` (no
989
+ control notebook exists on the hub β€” only this file and
990
+ `mini-beatrix-1/beatrix_curriculum.ipynb`); `AlephLMConfig.hub_chunk` defaults to
991
+ 128 while the module's own default is 256 (`presets.py:52`;
992
+ `attention.py:83`) β€” the config governs, and 2s sets 256.
993
+ - `governor_hits_cum` resets per session; the manifests record no
994
+ train_config or package version β€” the mother null's "identical
995
+ TrainConfig" is a git fact, and the craft's birth version 0.8.1 is
996
+ inferred from commit dates (run1 created 08-27T00:35Z between 0.8.1
997
+ on 08-26 and 0.8.2 on 08-29; the revival report records 0.8.2; the hub
998
+ driver `beatrix_v2_colab.ipynb` pins `geolip-alephllm @2a902b7`, the
999
+ 0.8.1 commit).
1000
+ - Owed instruments: the per-layer hidden-erank census of both crafts at
1001
+ the full-scale boundaries; the falsifier S-1803-F (Tier 0 on the 4090;
1002
+ a 40M twin screen with sdpa + QK-norm / LR/3 / dense-FFN / positions
1003
+ on Adam arms; pivot = resume the control at 20,219 through s0 at LR/3
1004
+ with QK-norm). Without it the general claim stays CANDIDATE.
1005
+ - `article_assets_ft2/make_charts_ft2.py` sources chart 01 from the hub
1006
+ but charts 02–05 from local scratch paths, and chart 01 pairs the
1007
+ struck s8 rows unless filtered.
1008
+
1009
+ ## Provenance discipline
1010
+
1011
+ Both crafts are resume-first: the manifest is embedded in the resume
1012
+ payload (atomic consistency), a crash never overwrites
1013
+ `resume/latest.pt` (`trainer.py:334-368`: any exception other than a
1014
+ manual stop leaves the hub state untouched), an interrupted session
1015
+ uploads before exit, and every phase boundary archives its own
1016
+ immutable resume. The revival is auditable from the prefix alone: the
1017
+ pre-write state (`resume/pre_revival_step24860.pt`), the op's
1018
+ provenance (`reports/v2/revival_step24860.json`, with version and
1019
+ commit), the pre- and post-write weights (`step_00024860` and
1020
+ `step_00024860_revived`), and the first live row after it
1021
+ (`curriculum_s1_step25561.json`). The struck s8 ledger stays public
1022
+ beside the cap that supersedes it, as run0 stays public under
1023
+ `archive/`, for the same reason the mother's rewind checkpoints do. The
1024
+ recipe is pinned from source, not from the manifests. The AutoModel
1025
+ vendors the model files verbatim and ships the mission-final weights
1026
+ re-prefixed; the training checkpoints, not the card, are the control's
1027
+ weights. Every refuted arm of the head-cultivation screen ships.
1028
+
1029
+ ## Pointers
1030
+
1031
+ - Canon: `canon/splat_aleph_battery.md` β€” CONTROL BOUNDARY 1–12
1032
+ (522–889), ENDPOINT VERDICT (891), CLAIM AUDIT (901–1006: the s8
1033
+ strike 930–937, MUON IS CLIP-BLIND 936, THE MOTHER NULL 955–968,
1034
+ PROVENANCE RIDER 1008), ROUND 5a/5d/5e, B2/B4/B5.
1035
+ - Statutes (`STATUTES_INDEX.md`): S-701 a control must be able to fail;
1036
+ S-801 supply law K ≀ 2D; S-1001 gate-zero deadlock; S-1002 born-null
1037
+ head self-burial 3/3; S-1003 BOUNDARY-WRITE; S-1201 optimizer family;
1038
+ S-1202 autocast einsum-promote; S-1801 the crossover; S-1802 the
1039
+ destabilised control (INSTANCE); S-1803 sdpa-inadequacy (CANDIDATE);
1040
+ S-1901 the specials plane.
1041
+ - Manual: `repos/alephllm.md` β€” the 0.7.0 era (1180–1330) and RIDERS
1042
+ 7–21b (1333–2081): pivot, launch health, specials and the run0
1043
+ archive, driver contract, the dead head, the revival trial, the
1044
+ boundary-write and promote, the head trend 14–14l, curriculum complete
1045
+ and the s8 artifact, the pre-chat baseline, mission complete, the
1046
+ ship-out, twin isolation, the pre-registered protocol.
1047
+ - Plans: `history/plans/2026-08-26_mini_beatrix_v2_shape.md` (the
1048
+ shape arithmetic), `2026-08-26_mini_beatrix_2s_mission.md` (the pivot,
1049
+ the anneal-timing ruling, the two-phase amendment),
1050
+ `2026-08-24_covariant_address_program.md` (the governor),
1051
+ `2026-09-02_sdpa_inadequacy_claim_held.md`,
1052
+ `2026-09-04_sdpa_falsifier_screen.md`.
1053
+ - Dockets: `history/dockets/2026-09-04_sdpa_claim_audit/`,
1054
+ `2026-09-01_ft2_article/` and `2026-09-01_article/tools/`
1055
+ (`writeback_cb*.py`, `writeback_endpoint.py`).
1056
+ - Article: `articles/article_beatrix_ft2.md`; charts under
1057
+ `article_assets_ft2/` on the training repo.
1058
+ - Code: AlephLLM HEAD 4f4b7d6 (0.8.7); the era's commits a63a29b
1059
+ (0.7.0) Β· 2be28dd (0.7.2) Β· 60ee496 (0.7.3) Β· 73218bd (0.7.5, the
1060
+ pivot) Β· 3ea3103 (0.8.0) Β· 2a902b7 (0.8.1) Β· d6e1854 (0.8.2) Β·
1061
+ 7d48328 (0.8.5) Β· af9319c (0.8.6) Β· b472815 (0.8.7). No git tags
1062
+ exist; versions are commit subjects.
1063
+ - The mother: `TECHNICAL.md` on the training repo (byte-identical local
1064
+ copy `articles/TECHNICAL_training_repo_beatrix.md`).