# mini-beatrix-3: stage arms fitted on the finished core Training of mini-beatrix-3 ended at step 245,674. On 2026-10-05 stage arms were fitted on that final core with the core frozen. This folder holds every arm those runs produced and the result file of every run. The arms packaged in [AbstractPhil/mini-beatrix-3](https://huggingface.co/AbstractPhil/mini-beatrix-3) are a subset, named at the end. An arm is a small detachable adapter (13.7M parameters, one module after each of the 32 blocks). Every arm here is bound to the final core in its bf16 weights and loads with [amoe-lora](https://github.com/AbstractEyes/amoe-lora). ## The runs Stages 1 to 4 are perspective, concepts, rule chains and arithmetic; stages 5 to 8 are cause and effect, try-fail-revise, the mixed stage and register. A run name ending in `A` or `B` is one of two seeds of the same recipe. | run | what was trained | arm seeds | data seed | |---|---|---|---| | solo, `fresh` | one new arm alone per stage, 800 steps, kept quiet on web text | 1 to 8, 101 to 104 | 20261005, 20261006 | | solo, `anchor` | one arm alone per stage, continued from the arm that was trained beside the trunk during the run (stages 1 to 4) | 1 to 4 | 20261005 | | solo, `pc_s1` | a repeat of an earlier single-arm control on stage 1 text (400 steps), run first to check the tool against a known result | 1 | 20260930 | | `solo_long/` | one solo arm for 3,200 steps with a reading every 400 (stage 3, stage 4), to see where an arm levels off | 3, 4 | 20261005 | | `solo_4000/` | one solo arm per stage at the arm length used in the certified ladder, 4,000 steps, kept quiet on web text | 1 to 8, 101 to 108 | 20261005, 20261006 | | `gXA`, `gXB` | the eight: the four arms of `gRA` / `gRB` switched on and frozen; new arms for stages 5, 6, 7 and 8 trained beside them one after the other, 4,000 steps each, each kept quiet on the stage text of every other arm | 5 to 8, 105 to 108 | 20261011, 20261012 | | `gEA`, `gEB` | all eight arms from scratch: frozen in turn as in `gS`, every arm kept quiet on every other arm's stage text from its first step, 4,000 steps per stage | 1 to 8, 101 to 108 | 20261013, 20261014 | | `gA`, `gB` | co-trained group: the arms of stages 1 to 4 attached one stage at a time, each new arm over the earlier ones, every attached arm training, 800 steps per stage | 1 to 4, 101 to 104 | 20261005, 20261006 | | `gFA`, `gFB` | the same group with the earlier arms switched on but frozen while the newest trains | 1 to 4, 101 to 104 | 20261005, 20261006 | | `gSA`, `gSB` | frozen in turn as in `gF`, and every arm kept quiet on the stage text of every other arm from its first step | 1 to 4, 101 to 104 | 20261005, 20261006 | | `gRA`, `gRB` | arms 1 and 2 of `gA` / `gB` (their fourth close) switched on and frozen; new arms for stages 3 and 4 trained beside them, one after the other, 800 steps each, kept quiet on the stage text of every other arm | 3 and 4, 103 and 104 | 20261007, 20261008 | | `routed_` | no arm trained: the four arms of a finished group under amoe-lora's routed dispatch, with only the dispatch keys trained (600 steps) | | 20261009 | Quiet means a penalty, at weight 2, on the divergence between the model with that arm switched off and the model with it on, measured on text that is not the arm's own. Every arm was kept quiet on ordinary web text. A group arm was also kept quiet on other arms' stage text: in `gA`, `gB`, `gFA` and `gFB` on the text of the arms attached so far, in `gSA`, `gSB` and for the new arms of `gRA`, `gRB` on the text of every other arm of the group from the first step. A run listed above that has no files here had not finished when this folder was last updated. ## Files | path | what it is | |---|---| | `group/anchors/_close_.safetensors` | the arm of stage `` in run `` at the close of the run's k-th stage | | `group/results/group_.json` | the run's measurements at every stage close: each stage's held-out text and web text with arms off and with the group on, each member's share inside the group and its reading alone, the stage test items, the probe suite, the training curve | | `group/results/routed_.json`, `group/anchors/routed_.dispatch.pt` | a routed reading of a finished group and its trained dispatch keys | | `solo/anchors/__seed_d_step.safetensors` | a solo arm at step 400 and at step 800 | | `solo/results/result___seed_d.json` | the solo run's measurements at steps 0, 200, 400 and 800 | | `solo_long/`, `solo_4000/` | the same two kinds of file for the 3,200-step and the 4,000-step solo runs | | `logs/` | the runs' own logs | All losses are bits per byte on held-out text. ## What the runs found Held-out stage text with every arm of the group on, two seeds: | | stage 1 | stage 2 | stage 3 | stage 4 | each arm also works alone | |---|---|---|---|---|---| | arms off | 0.554 | 0.721 | 0.930 | 1.234 | | | a solo arm, one at a time | 0.081 | 0.120 | 0.372 | 0.389 | yes | | `gA` / `gB` | 0.048 / 0.048 | 0.111 / 0.121 | 0.310 / 0.309 | 0.464 / 0.490 | no | | `gFA` / `gFB` | 0.081 / 0.080 | 0.109 / 0.109 | 0.358 / 0.356 | 0.367 / 0.368 | in part | | `gSA` / `gSB` | 0.081 / 0.080 | 0.120 / 0.120 | 0.376 / 0.377 | 0.392 / 0.392 | yes | | `gRA` / `gRB` | 0.048 / 0.048 | 0.112 / 0.121 | 0.224 / 0.222 | 0.338 / 0.338 | no | - A solo arm recovers its own stage's text and leaves web text alone. Four solo arms switched on together do not work: three of the four stage texts read worse than with no arm. - The co-trained group works with every arm on, but it does not divide its work one stage to an arm: the arm attached first keeps training under every later stage and takes up most of their text, and the arms attached last hold almost nothing. - Freezing the earlier arms gives each arm most of its own stage. Keeping every arm quiet on every other arm's text from its first step completes it: each arm then holds 94 to 99 percent of its stage, reads alone like a solo arm, and the four work together. - A frozen pair from the co-trained group with two new arms trained beside it reads best with all four on. Its first arm does most of the work on every stage and the two new arms add to it; they do little alone. - Under the routed dispatch the same arms read worse than stacked on every stage, for both kinds of group. - No arm is trained to its limit at 800 steps: a solo arm's stage text is still falling quickly there. The first two arms of `gRA` / `gRB` carry 3,200 and 2,400 steps, the others 800. In every group, web text moved by at most 0.002 bits per byte. ## Packaged in the model repository | in `AbstractPhil/mini-beatrix-3` | from this folder | |---|---| | `arms/stages-1-4/.safetensors` | `group/anchors/gRA_close4_.safetensors` | | `arms/solo/.safetensors` | `solo/anchors/_fresh_seed<1 to 4>_d20261005_step800.safetensors` | The arms that were trained beside the trunk during the run itself are in `mini-beatrix-3/arms/` of this repository. They are bound to earlier trunk states, not to the final core.