surface arm clip/mse_solo_o0_norm: anchors (every close), stats, results, log, readout (the second home until the publishing convention is settled; Phil 2026-10-08)
Browse files- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close1_s12_clip.safetensors +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close2_s12_clip.safetensors +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close3_s12_clip.safetensors +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close4_s12_clip.safetensors +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/manifest.json +20 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/readout.md +127 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/results.json +0 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/s12_clip.safetensors +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/stats.pt +3 -0
- mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/train.log +160 -0
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close1_s12_clip.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2bb1a2a353c98f5d193930adbe5616efd6d59d4a600ea16392772d9d74bbec44
|
| 3 |
+
size 54819336
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close2_s12_clip.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5af797fc8c9900a042b0b5599d20854afc47e2f703d2b28face630c9525cff94
|
| 3 |
+
size 54819336
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close3_s12_clip.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4b4a0049444bbed72f61cf99fed7fa8054b9154c564e213fdecc613a8f8c5bd1
|
| 3 |
+
size 54819336
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/closes/sarm_clip_mse_solo_o0_norm_close4_s12_clip.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ccda78eeaa5b2ae582c8632a735cf8994e4c6560a07cf3b98a7df6ddd29330d0
|
| 3 |
+
size 54819336
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/manifest.json
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"tag": "sarm_clip_mse_solo_o0_norm",
|
| 3 |
+
"run": "mse_solo_o0_norm",
|
| 4 |
+
"convention": "clip",
|
| 5 |
+
"member": "s12_clip",
|
| 6 |
+
"final_anchor": "sarm_clip_mse_solo_o0_norm_close4_s12_clip.safetensors",
|
| 7 |
+
"closes": [
|
| 8 |
+
"sarm_clip_mse_solo_o0_norm_close1_s12_clip.safetensors",
|
| 9 |
+
"sarm_clip_mse_solo_o0_norm_close2_s12_clip.safetensors",
|
| 10 |
+
"sarm_clip_mse_solo_o0_norm_close3_s12_clip.safetensors",
|
| 11 |
+
"sarm_clip_mse_solo_o0_norm_close4_s12_clip.safetensors"
|
| 12 |
+
],
|
| 13 |
+
"finished": true,
|
| 14 |
+
"form": "mse",
|
| 15 |
+
"base": "solo",
|
| 16 |
+
"steps": 4000,
|
| 17 |
+
"trunk_step": 245674,
|
| 18 |
+
"shipped": "2026-10-09 13:10 local",
|
| 19 |
+
"note": "every close's anchor ships (mids included); the final close's anchor is the run's s12_clip.safetensors"
|
| 20 |
+
}
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/readout.md
ADDED
|
@@ -0,0 +1,127 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# The surface arm's readout: clip-vit-large-patch14's spelling (CLIP's convention)
|
| 2 |
+
|
| 3 |
+
A surface arm makes the trunk read a tokenizer's spelling of a text as it reads the text's own bytes. Every number below is read on the dual-extraction read's held-out captions (512 of each COCO draw, never trained on; draw 2 is the repeat), one document a row, at every token's closing byte, as whitened Procrustes alignment (1 = the same geometry up to a rotation; two folds by caption). The pre-registered bars are in the plan; B3 and B4 (against Qwen's own states, the mood axis) are the instrument's read.
|
| 4 |
+
|
| 5 |
+
## The runs
|
| 6 |
+
|
| 7 |
+
- sarm_clip_mse_solo_o0_norm: per-site MSE; alone on the bare trunk; blocks 0-31; 4000 of 4000 steps; seed offset 0
|
| 8 |
+
|
| 9 |
+
## The verdicts (the final close of each run)
|
| 10 |
+
|
| 11 |
+
### sarm_clip_mse_solo_o0_norm
|
| 12 |
+
|
| 13 |
+
- B1 the gap: NOT MET over every served block >= 12 (the lowest 0.786 at block 12 on draw 1, 0.787 at block 12 on draw 2; bar 0.85); over blocks 12-28 alone: NOT MET
|
| 14 |
+
- B2 silence, whitened: NOT MET (the lowest A alignment 0.9814 at block 30 on draw 1, 0.9825 on draw 2; bar 0.995)
|
| 15 |
+
- B2 silence, in bits: MET (web +0.00019 bpb; bar +0.012)
|
| 16 |
+
|
| 17 |
+
## The closes
|
| 18 |
+
|
| 19 |
+
### sarm_clip_mse_solo_o0_norm
|
| 20 |
+
|
| 21 |
+
| close | step | B1 lowest (draw 1 / 2) | B1 at 12 / 20 / 28 (draw 1) | B2 lowest (draw 1 / 2) | web bits | gate mean / widest |
|
| 22 |
+
|---|---|---|---|---|---|---|
|
| 23 |
+
| 0 | 0 | 0.294 / 0.303 | 0.431 / 0.378 / 0.316 | 1.0000 / 1.0000 | +0.00000 | 0.047 / 0.047 |
|
| 24 |
+
| 1 | 1000 | 0.673 / 0.674 | 0.673 / 0.764 / 0.774 | 0.9662 / 0.9662 | +0.00010 | 0.139 / 0.166 |
|
| 25 |
+
| 2 | 2000 | 0.722 / 0.722 | 0.722 / 0.843 / 0.855 | 0.9797 / 0.9782 | +0.00051 | 0.205 / 0.283 |
|
| 26 |
+
| 3 | 3000 | 0.752 / 0.754 | 0.752 / 0.876 / 0.887 | 0.9822 / 0.9814 | +0.00018 | 0.288 / 0.542 |
|
| 27 |
+
| 4 | 4000 | 0.786 / 0.787 | 0.786 / 0.901 / 0.910 | 0.9814 / 0.9825 | +0.00019 | 0.369 / 0.694 |
|
| 28 |
+
|
| 29 |
+
## Every block at the final close
|
| 30 |
+
|
| 31 |
+
### sarm_clip_mse_solo_o0_norm, draw1 (512 captions, 4,871 sites), close 4 at step 4000
|
| 32 |
+
|
| 33 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 34 |
+
|---|---|---|---|---|---|---|
|
| 35 |
+
| 0 | 0.239 | 0.233 | 0.232 | 0.9997 | 1.264 | 0.165 |
|
| 36 |
+
| 4 | 0.509 | 0.594 | 0.598 | 0.9921 | 0.488 | 0.681 |
|
| 37 |
+
| 8 | 0.478 | 0.676 | 0.681 | 0.9928 | 0.385 | 0.774 |
|
| 38 |
+
| 12 | 0.431 | 0.786 | 0.792 | 0.9942 | 0.305 | 0.841 |
|
| 39 |
+
| 16 | 0.401 | 0.867 | 0.869 | 0.9904 | 0.209 | 0.896 |
|
| 40 |
+
| 18 | 0.390 | 0.889 | 0.890 | 0.9903 | 0.178 | 0.913 |
|
| 41 |
+
| 20 | 0.378 | 0.901 | 0.903 | 0.9913 | 0.157 | 0.924 |
|
| 42 |
+
| 22 | 0.393 | 0.914 | 0.915 | 0.9924 | 0.136 | 0.937 |
|
| 43 |
+
| 24 | 0.390 | 0.917 | 0.918 | 0.9889 | 0.127 | 0.941 |
|
| 44 |
+
| 26 | 0.356 | 0.917 | 0.918 | 0.9920 | 0.121 | 0.943 |
|
| 45 |
+
| 28 | 0.316 | 0.910 | 0.911 | 0.9886 | 0.126 | 0.938 |
|
| 46 |
+
| 29 | 0.300 | 0.906 | 0.906 | 0.9869 | 0.130 | 0.934 |
|
| 47 |
+
| 30 | 0.294 | 0.898 | 0.898 | 0.9814 | 0.140 | 0.929 |
|
| 48 |
+
| 31 | 0.298 | 0.892 | 0.892 | 0.9843 | 0.144 | 0.923 |
|
| 49 |
+
|
| 50 |
+
### sarm_clip_mse_solo_o0_norm, draw2 (512 captions, 5,023 sites), close 4 at step 4000
|
| 51 |
+
|
| 52 |
+
| block | gap before | armed B vs masked A | armed B vs armed A | A armed vs masked | mse | cos |
|
| 53 |
+
|---|---|---|---|---|---|---|
|
| 54 |
+
| 0 | 0.260 | 0.253 | 0.252 | 0.9997 | 1.258 | 0.163 |
|
| 55 |
+
| 4 | 0.510 | 0.604 | 0.608 | 0.9937 | 0.483 | 0.681 |
|
| 56 |
+
| 8 | 0.485 | 0.684 | 0.689 | 0.9956 | 0.372 | 0.779 |
|
| 57 |
+
| 12 | 0.438 | 0.787 | 0.794 | 0.9921 | 0.292 | 0.845 |
|
| 58 |
+
| 16 | 0.413 | 0.871 | 0.874 | 0.9920 | 0.199 | 0.900 |
|
| 59 |
+
| 18 | 0.399 | 0.893 | 0.895 | 0.9908 | 0.169 | 0.916 |
|
| 60 |
+
| 20 | 0.384 | 0.901 | 0.902 | 0.9903 | 0.151 | 0.926 |
|
| 61 |
+
| 22 | 0.400 | 0.914 | 0.916 | 0.9924 | 0.133 | 0.938 |
|
| 62 |
+
| 24 | 0.395 | 0.917 | 0.919 | 0.9910 | 0.125 | 0.941 |
|
| 63 |
+
| 26 | 0.359 | 0.917 | 0.917 | 0.9866 | 0.120 | 0.943 |
|
| 64 |
+
| 28 | 0.320 | 0.911 | 0.912 | 0.9907 | 0.125 | 0.937 |
|
| 65 |
+
| 29 | 0.305 | 0.906 | 0.908 | 0.9869 | 0.129 | 0.934 |
|
| 66 |
+
| 30 | 0.303 | 0.896 | 0.898 | 0.9878 | 0.142 | 0.928 |
|
| 67 |
+
| 31 | 0.308 | 0.892 | 0.894 | 0.9825 | 0.150 | 0.923 |
|
| 68 |
+
|
| 69 |
+
The member's share in bits per byte (all on minus the member masked): web +0.00019
|
| 70 |
+
|
| 71 |
+
## The training curves
|
| 72 |
+
|
| 73 |
+
### sarm_clip_mse_solo_o0_norm
|
| 74 |
+
|
| 75 |
+
| step | task loss | mse | cos | InfoNCE | quiet on A | quiet off-domain |
|
| 76 |
+
|---|---|---|---|---|---|---|
|
| 77 |
+
| 1 | 2.1302 | 2.130 | 0.120 | None | 0.00000 | 0.00000 |
|
| 78 |
+
| 50 | 2.1234 | 2.123 | 0.118 | None | 0.00030 | 0.00023 |
|
| 79 |
+
| 250 | 1.9350 | 1.935 | 0.159 | None | 0.00156 | 0.00106 |
|
| 80 |
+
| 500 | 1.3417 | 1.342 | 0.365 | None | 0.00155 | 0.00046 |
|
| 81 |
+
| 1000 | 0.8937 | 0.894 | 0.554 | None | 0.00120 | 0.00064 |
|
| 82 |
+
| 1500 | 0.6929 | 0.693 | 0.641 | None | 0.00105 | 0.00065 |
|
| 83 |
+
| 2000 | 0.5601 | 0.560 | 0.706 | None | 0.00113 | 0.00072 |
|
| 84 |
+
| 2500 | 0.4743 | 0.474 | 0.748 | None | 0.00117 | 0.00075 |
|
| 85 |
+
| 3000 | 0.4189 | 0.419 | 0.774 | None | 0.00117 | 0.00076 |
|
| 86 |
+
| 3500 | 0.3606 | 0.361 | 0.803 | None | 0.00116 | 0.00074 |
|
| 87 |
+
| 4000 | 0.3218 | 0.322 | 0.822 | None | 0.00109 | 0.00065 |
|
| 88 |
+
| 1 | 2.1302 | 2.130 | 0.120 | None | 0.00000 | 0.00000 |
|
| 89 |
+
| 50 | 2.1234 | 2.123 | 0.118 | None | 0.00030 | 0.00023 |
|
| 90 |
+
| 250 | 1.9350 | 1.935 | 0.159 | None | 0.00156 | 0.00106 |
|
| 91 |
+
| 500 | 1.3417 | 1.342 | 0.365 | None | 0.00155 | 0.00046 |
|
| 92 |
+
| 1000 | 0.8937 | 0.894 | 0.554 | None | 0.00120 | 0.00064 |
|
| 93 |
+
| 1500 | 0.6929 | 0.693 | 0.641 | None | 0.00105 | 0.00065 |
|
| 94 |
+
| 2000 | 0.5601 | 0.560 | 0.706 | None | 0.00113 | 0.00072 |
|
| 95 |
+
| 2500 | 0.4743 | 0.474 | 0.748 | None | 0.00117 | 0.00075 |
|
| 96 |
+
| 3000 | 0.4189 | 0.419 | 0.774 | None | 0.00117 | 0.00076 |
|
| 97 |
+
| 3500 | 0.3606 | 0.361 | 0.803 | None | 0.00116 | 0.00074 |
|
| 98 |
+
| 4000 | 0.3218 | 0.322 | 0.822 | None | 0.00109 | 0.00065 |
|
| 99 |
+
| 1 | 2.1302 | 2.130 | 0.120 | None | 0.00000 | 0.00000 |
|
| 100 |
+
| 50 | 2.1234 | 2.123 | 0.118 | None | 0.00030 | 0.00023 |
|
| 101 |
+
| 250 | 1.9350 | 1.935 | 0.159 | None | 0.00156 | 0.00106 |
|
| 102 |
+
| 500 | 1.3417 | 1.342 | 0.365 | None | 0.00155 | 0.00046 |
|
| 103 |
+
| 1000 | 0.8937 | 0.894 | 0.554 | None | 0.00120 | 0.00064 |
|
| 104 |
+
| 1500 | 0.6929 | 0.693 | 0.641 | None | 0.00105 | 0.00065 |
|
| 105 |
+
| 2000 | 0.5601 | 0.560 | 0.706 | None | 0.00113 | 0.00072 |
|
| 106 |
+
| 2500 | 0.4743 | 0.474 | 0.748 | None | 0.00117 | 0.00075 |
|
| 107 |
+
| 3000 | 0.4189 | 0.419 | 0.774 | None | 0.00117 | 0.00076 |
|
| 108 |
+
| 3500 | 0.3606 | 0.361 | 0.803 | None | 0.00116 | 0.00074 |
|
| 109 |
+
| 4000 | 0.3218 | 0.322 | 0.822 | None | 0.00109 | 0.00065 |
|
| 110 |
+
| 1 | 2.1302 | 2.130 | 0.120 | None | 0.00000 | 0.00000 |
|
| 111 |
+
| 50 | 2.1234 | 2.123 | 0.118 | None | 0.00030 | 0.00023 |
|
| 112 |
+
| 250 | 1.9350 | 1.935 | 0.159 | None | 0.00156 | 0.00106 |
|
| 113 |
+
| 500 | 1.3417 | 1.342 | 0.365 | None | 0.00155 | 0.00046 |
|
| 114 |
+
| 1000 | 0.8937 | 0.894 | 0.554 | None | 0.00120 | 0.00064 |
|
| 115 |
+
| 1500 | 0.6929 | 0.693 | 0.641 | None | 0.00105 | 0.00065 |
|
| 116 |
+
| 2000 | 0.5601 | 0.560 | 0.706 | None | 0.00113 | 0.00072 |
|
| 117 |
+
| 2500 | 0.4743 | 0.474 | 0.748 | None | 0.00117 | 0.00075 |
|
| 118 |
+
| 3000 | 0.4189 | 0.419 | 0.774 | None | 0.00117 | 0.00076 |
|
| 119 |
+
| 3500 | 0.3606 | 0.361 | 0.803 | None | 0.00116 | 0.00074 |
|
| 120 |
+
| 4000 | 0.3218 | 0.322 | 0.822 | None | 0.00109 | 0.00065 |
|
| 121 |
+
|
| 122 |
+
## The recipe
|
| 123 |
+
|
| 124 |
+
- per step: the teacher (member masked) reads the A rows: block outputs at the sites + logits; the armed model reads the B rows: per-site MSE of standardized LayerNorm'd block outputs against the teacher's at each token's closing byte, mean over served blocks; quiet on A: lambda x KL(teacher || armed) on the A rows; quiet off-domain: lambda x KL(member masked || armed) on one web row + one partner row
|
| 125 |
+
- lambda 2.0; pure Adam lr 0.001 wd 0; 2 x 4096-byte paired rows a step; the arm LAWFUL_16x8 at every block ({'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}); the trunk mini-beatrix-3 at step 245,674; alephllm 0.10.7, amoe-lora 0.2.11, torch 2.8.0+cu128
|
| 126 |
+
- the surface: clip-vit-large-patch14's spelling (CLIP's convention; sha256 4a07de0310c7cbea..., vocab 49,408; A = the text normalized; sites = tokens); documents = one caption-like field of a caption-pack row (caption, short, long, tags, medium), sources drawn by {'dff': 0.161, 'scf': 0.161, 'danbooru': 0.164, 'cc12m': 0.406, 'coco': 0.108}
|
| 127 |
+
- the test set: the first 512 captions of each draw of coco_caps_2x2048.json (every caption of the file kept out of training)
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/results.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/s12_clip.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ccda78eeaa5b2ae582c8632a735cf8994e4c6560a07cf3b98a7df6ddd29330d0
|
| 3 |
+
size 54819336
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/stats.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f3e719e71c6f0b1a956c2bfd653721274eb013d18e5e4525a3f919868d0f15f7
|
| 3 |
+
size 264365
|
mini-beatrix-3/arm_refit/surface/clip/mse_solo_o0_norm/train.log
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[ 0.8s] THE JOB: sarm_clip_mse_solo_o0_norm: the surface arm s12_clip (clip spelling by openai/clip-vit-large-patch14; A = the text normalized; sites = tokens) alone on the bare trunk; form mse; blocks all; 4000 steps, a close every 1000; lambda 2.0; pure Adam lr 0.001; the compiled chain with recompute; card 30.5 GB with 30.4 GB free; host RSS 0.6 GB
|
| 2 |
+
Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.
|
| 3 |
+
[ 17.6s] mini-beatrix-3 @ 245,674: trunk loaded from /workspace/qarm/dl/mini-beatrix-3/checkpoints/step_00245674.safetensors and frozen (376.0M); alephllm 0.10.7 at /workspace/qarm/alephllm/geolip/alephllm; amoe-lora 0.2.11; torch 2.8.0+cu128
|
| 4 |
+
[ 18.0s] tokenizer openai/clip-vit-large-patch14 (clip spelling; vocab 49408; sha256 4a07de0310c7cbea): 'a taco truck' -> ['a</w>', 'taco</w>', 'truck</w>']; A 12 bytes, B 22 bytes; sites [(1, 6, 13)]
|
| 5 |
+
[ 18.0s] the read's captions: 512 of each draw of /workspace/qarm/dl/coco_caps_2x2048.json (4,090 captions of both draws kept out of training)
|
| 6 |
+
[ 31.2s] caption pack (/workspace/qarm/dl/mini-beatrix-3/caption_pack/pack_v2, train): dff 121,153 rows, scf 42,080 rows, danbooru 603,026 rows, cc12m 1,242,176 rows, coco 19,227 rows; host RSS 3.6 GB
|
| 7 |
+
[ 31.3s] streams: web text; partners none (two web rows a quiet chunk); held-out sets ['web'] x 8 batches
|
| 8 |
+
[ 31.4s] read rows draw1: 512 captions (0 unspellable), A rows (512, 99), B rows (512, 169), 4,871 sites
|
| 9 |
+
[ 31.5s] read rows draw2: 512 captions (0 unspellable), A rows (512, 151), B rows (512, 246), 5,023 sites
|
| 10 |
+
[arms] attached 's12_clip' for phase 'clip_spelling': 13.69M params x 32 blocks, lambda 2.0, spec {'n_slots': 16, 'K': 16, 'D': 8, 'hidden': 256}, recompute on, compile on
|
| 11 |
+
[transformers] Token indices sequence length is longer than the specified maximum sequence length for this model (98 > 77). Running this sequence through the model will result in indexing errors
|
| 12 |
+
[ 43.1s] BIRTH: the member attached over the trunk: web rows read exactly as before; the hand-run trunk against the model's forward: armed 0.0e+00, masked 0.0e+00 (gates: 0.0)
|
| 13 |
+
[ 46.4s] standardization statistics from 8,173 teacher sites on 8 paired batches (a mean and a spread per block and dimension); sigma range at block 20 0.732..1.104; saved /workspace/qarm/out/anchors/sarm_clip_mse_solo_o0_norm_stats.pt
|
| 14 |
+
[ 253.8s] CLOSE 0 (step 0): the member's alignment read on draw 1 (512 captions, 4,871 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 15 |
+
[ 253.8s] block 8: 0.478 -> 0.478 / 0.478; A 1.0000; mse 1.927 cos 0.174
|
| 16 |
+
[ 253.8s] block 12: 0.431 -> 0.431 / 0.431; A 1.0000; mse 1.809 cos 0.161
|
| 17 |
+
[ 253.8s] block 16: 0.401 -> 0.401 / 0.401; A 1.0000; mse 1.708 cos 0.147
|
| 18 |
+
[ 253.8s] block 18: 0.390 -> 0.390 / 0.390; A 1.0000; mse 1.666 cos 0.149
|
| 19 |
+
[ 253.8s] block 20: 0.378 -> 0.378 / 0.378; A 1.0000; mse 1.697 cos 0.141
|
| 20 |
+
[ 253.8s] block 24: 0.390 -> 0.390 / 0.390; A 1.0000; mse 1.923 cos 0.150
|
| 21 |
+
[ 253.8s] block 28: 0.316 -> 0.316 / 0.316; A 1.0000; mse 2.139 cos 0.113
|
| 22 |
+
[ 253.8s] block 31: 0.298 -> 0.298 / 0.298; A 1.0000; mse 1.722 cos 0.095
|
| 23 |
+
[ 253.8s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.438 / 0.413 / 0.384 / 0.395 / 0.320
|
| 24 |
+
[ 253.8s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.294 against the bar .85; B2: the lowest A alignment 1.0000 against .995; fineweb share +0.00000 +- 0.00000 against +.012
|
| 25 |
+
[ 253.8s] gate mean 0.047 / widest 0.047; compiled chain: 2 cache entries (limit 256), 0 limit warnings; docs drawn 266 (skipped {'test': 2, 'spelling': 5, 'empty': 0, 'short': 0}); card peak 2.3 GB; host RSS 10.1 GB
|
| 26 |
+
[stream] opening 'fineweb-edu' (HuggingFaceFW/fineweb-edu) …
|
| 27 |
+
[ 261.4s] step 1/4000: task 2.1302 (mse 2.130 cos 0.120); quiet A 0.00000 off 0.00000; 1,021 sites; 7.60 s/step; the job: 4.4 min spent, about 506.6 min of steps left; card peak 13.5 GB
|
| 28 |
+
[ 382.1s] step 50/4000: task 2.1234 (mse 2.123 cos 0.118); quiet A 0.00030 off 0.00023; 1,035 sites; 2.57 s/step; the job: 6.4 min spent, about 169.0 min of steps left; card peak 13.6 GB
|
| 29 |
+
[ 505.2s] step 100/4000: task 2.1043 (mse 2.104 cos 0.120); quiet A 0.00039 off 0.00029; 977 sites; 2.51 s/step; the job: 8.4 min spent, about 163.4 min of steps left; card peak 13.6 GB
|
| 30 |
+
[ 628.3s] step 150/4000: task 2.0637 (mse 2.064 cos 0.128); quiet A 0.00057 off 0.00038; 1,034 sites; 2.50 s/step; the job: 10.5 min spent, about 160.2 min of steps left; card peak 13.6 GB
|
| 31 |
+
[ 751.3s] step 200/4000: task 2.0092 (mse 2.009 cos 0.142); quiet A 0.00100 off 0.00063; 1,011 sites; 2.49 s/step; the job: 12.5 min spent, about 157.5 min of steps left; card peak 13.6 GB
|
| 32 |
+
[ 874.4s] step 250/4000: task 1.9350 (mse 1.935 cos 0.159); quiet A 0.00156 off 0.00106; 1,027 sites; 2.48 s/step; the job: 14.6 min spent, about 155.2 min of steps left; card peak 13.6 GB
|
| 33 |
+
[ 997.3s] step 300/4000: task 1.8374 (mse 1.837 cos 0.190); quiet A 0.00201 off 0.00083; 1,019 sites; 2.48 s/step; the job: 16.6 min spent, about 152.8 min of steps left; card peak 13.6 GB
|
| 34 |
+
[ 1120.7s] step 350/4000: task 1.7168 (mse 1.717 cos 0.227); quiet A 0.00217 off 0.00059; 1,024 sites; 2.48 s/step; the job: 18.7 min spent, about 150.7 min of steps left; card peak 13.6 GB
|
| 35 |
+
[ 1243.7s] step 400/4000: task 1.5771 (mse 1.577 cos 0.270); quiet A 0.00184 off 0.00052; 1,023 sites; 2.47 s/step; the job: 20.7 min spent, about 148.5 min of steps left; card peak 13.6 GB
|
| 36 |
+
[ 1366.7s] step 450/4000: task 1.4370 (mse 1.437 cos 0.324); quiet A 0.00167 off 0.00046; 1,038 sites; 2.47 s/step; the job: 22.8 min spent, about 146.3 min of steps left; card peak 13.6 GB
|
| 37 |
+
[ 1489.7s] step 500/4000: task 1.3417 (mse 1.342 cos 0.365); quiet A 0.00155 off 0.00046; 1,030 sites; 2.47 s/step; the job: 24.8 min spent, about 144.2 min of steps left; card peak 13.6 GB
|
| 38 |
+
[ 1612.7s] step 550/4000: task 1.2620 (mse 1.262 cos 0.401); quiet A 0.00136 off 0.00052; 1,045 sites; 2.47 s/step; the job: 26.9 min spent, about 142.1 min of steps left; card peak 13.6 GB
|
| 39 |
+
[ 1735.7s] step 600/4000: task 1.1939 (mse 1.194 cos 0.430); quiet A 0.00128 off 0.00054; 1,019 sites; 2.47 s/step; the job: 28.9 min spent, about 140.0 min of steps left; card peak 13.6 GB
|
| 40 |
+
[ 1859.0s] step 650/4000: task 1.1369 (mse 1.137 cos 0.455); quiet A 0.00120 off 0.00052; 1,003 sites; 2.47 s/step; the job: 31.0 min spent, about 137.9 min of steps left; card peak 13.6 GB
|
| 41 |
+
[ 1982.1s] step 700/4000: task 1.0864 (mse 1.086 cos 0.475); quiet A 0.00117 off 0.00054; 1,035 sites; 2.47 s/step; the job: 33.0 min spent, about 135.8 min of steps left; card peak 13.6 GB
|
| 42 |
+
[ 2105.2s] step 750/4000: task 1.0435 (mse 1.043 cos 0.494); quiet A 0.00115 off 0.00058; 1,035 sites; 2.47 s/step; the job: 35.1 min spent, about 133.7 min of steps left; card peak 13.6 GB
|
| 43 |
+
[ 2228.2s] step 800/4000: task 1.0084 (mse 1.008 cos 0.508); quiet A 0.00113 off 0.00058; 1,013 sites; 2.47 s/step; the job: 37.1 min spent, about 131.6 min of steps left; card peak 13.6 GB
|
| 44 |
+
[ 2351.2s] step 850/4000: task 0.9759 (mse 0.976 cos 0.521); quiet A 0.00120 off 0.00062; 1,021 sites; 2.47 s/step; the job: 39.2 min spent, about 129.5 min of steps left; card peak 13.6 GB
|
| 45 |
+
[ 2474.4s] step 900/4000: task 0.9491 (mse 0.949 cos 0.533); quiet A 0.00122 off 0.00070; 1,038 sites; 2.47 s/step; the job: 41.2 min spent, about 127.5 min of steps left; card peak 13.6 GB
|
| 46 |
+
[ 2597.3s] step 950/4000: task 0.9214 (mse 0.921 cos 0.544); quiet A 0.00120 off 0.00062; 1,022 sites; 2.47 s/step; the job: 43.3 min spent, about 125.4 min of steps left; card peak 13.6 GB
|
| 47 |
+
[ 2720.2s] step 1000/4000: task 0.8937 (mse 0.894 cos 0.554); quiet A 0.00120 off 0.00064; 1,012 sites; 2.47 s/step; the job: 45.3 min spent, about 123.3 min of steps left; card peak 13.6 GB
|
| 48 |
+
[ 2720.5s] anchor saved: /workspace/qarm/out/anchors/sarm_clip_mse_solo_o0_norm_close1_s12_clip.safetensors
|
| 49 |
+
[ 2875.7s] CLOSE 1 (step 1000): the member's alignment read on draw 1 (512 captions, 4,871 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 50 |
+
[ 2875.7s] block 8: 0.478 -> 0.603 / 0.609; A 0.9851; mse 0.853 cos 0.511
|
| 51 |
+
[ 2875.7s] block 12: 0.431 -> 0.673 / 0.682; A 0.9789; mse 0.702 cos 0.617
|
| 52 |
+
[ 2875.7s] block 16: 0.401 -> 0.717 / 0.721; A 0.9795; mse 0.568 cos 0.700
|
| 53 |
+
[ 2875.7s] block 18: 0.390 -> 0.743 / 0.748; A 0.9783; mse 0.516 cos 0.731
|
| 54 |
+
[ 2875.7s] block 20: 0.378 -> 0.764 / 0.768; A 0.9774; mse 0.467 cos 0.761
|
| 55 |
+
[ 2875.7s] block 24: 0.390 -> 0.794 / 0.802; A 0.9742; mse 0.393 cos 0.808
|
| 56 |
+
[ 2875.7s] block 28: 0.316 -> 0.774 / 0.782; A 0.9734; mse 0.387 cos 0.798
|
| 57 |
+
[ 2875.7s] block 31: 0.298 -> 0.745 / 0.755; A 0.9673; mse 0.398 cos 0.759
|
| 58 |
+
[ 2875.7s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.674 / 0.715 / 0.763 / 0.791 / 0.775
|
| 59 |
+
[ 2875.7s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.673 against the bar .85; B2: the lowest A alignment 0.9662 against .995; fineweb share +0.00010 +- 0.00016 against +.012
|
| 60 |
+
[ 2875.7s] gate mean 0.139 / widest 0.166; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 32,969 (skipped {'test': 45, 'spelling': 549, 'empty': 0, 'short': 0}); card peak 13.6 GB; host RSS 13.0 GB
|
| 61 |
+
[ 2998.8s] step 1050/4000: task 0.8681 (mse 0.868 cos 0.563); quiet A 0.00118 off 0.00066; 1,006 sites; 2.46 s/step; the job: 50.0 min spent, about 121.3 min of steps left; card peak 13.6 GB
|
| 62 |
+
[ 3121.7s] step 1100/4000: task 0.8453 (mse 0.845 cos 0.572); quiet A 0.00115 off 0.00058; 1,066 sites; 2.46 s/step; the job: 52.0 min spent, about 119.2 min of steps left; card peak 13.6 GB
|
| 63 |
+
[ 3244.6s] step 1150/4000: task 0.8230 (mse 0.823 cos 0.582); quiet A 0.00116 off 0.00058; 1,019 sites; 2.46 s/step; the job: 54.1 min spent, about 117.1 min of steps left; card peak 13.6 GB
|
| 64 |
+
[ 3367.8s] step 1200/4000: task 0.8022 (mse 0.802 cos 0.591); quiet A 0.00116 off 0.00061; 1,038 sites; 2.46 s/step; the job: 56.1 min spent, about 115.1 min of steps left; card peak 13.6 GB
|
| 65 |
+
[ 3490.7s] step 1250/4000: task 0.7830 (mse 0.783 cos 0.599); quiet A 0.00114 off 0.00063; 991 sites; 2.46 s/step; the job: 58.2 min spent, about 113.0 min of steps left; card peak 13.6 GB
|
| 66 |
+
[ 3613.5s] step 1300/4000: task 0.7690 (mse 0.769 cos 0.606); quiet A 0.00109 off 0.00058; 1,030 sites; 2.46 s/step; the job: 60.2 min spent, about 110.9 min of steps left; card peak 13.6 GB
|
| 67 |
+
[ 3736.4s] step 1350/4000: task 0.7444 (mse 0.744 cos 0.616); quiet A 0.00112 off 0.00059; 990 sites; 2.46 s/step; the job: 62.3 min spent, about 108.8 min of steps left; card peak 13.6 GB
|
| 68 |
+
[ 3859.5s] step 1400/4000: task 0.7299 (mse 0.730 cos 0.623); quiet A 0.00107 off 0.00060; 1,007 sites; 2.46 s/step; the job: 64.3 min spent, about 106.8 min of steps left; card peak 13.6 GB
|
| 69 |
+
[ 3982.8s] step 1450/4000: task 0.7138 (mse 0.714 cos 0.631); quiet A 0.00110 off 0.00060; 1,047 sites; 2.46 s/step; the job: 66.4 min spent, about 104.7 min of steps left; card peak 13.6 GB
|
| 70 |
+
[ 4106.2s] step 1500/4000: task 0.6929 (mse 0.693 cos 0.641); quiet A 0.00105 off 0.00065; 1,011 sites; 2.46 s/step; the job: 68.4 min spent, about 102.7 min of steps left; card peak 13.6 GB
|
| 71 |
+
[ 4229.2s] step 1550/4000: task 0.6782 (mse 0.678 cos 0.648); quiet A 0.00106 off 0.00062; 1,012 sites; 2.46 s/step; the job: 70.5 min spent, about 100.6 min of steps left; card peak 13.6 GB
|
| 72 |
+
[ 4352.4s] step 1600/4000: task 0.6614 (mse 0.661 cos 0.655); quiet A 0.00102 off 0.00057; 1,042 sites; 2.46 s/step; the job: 72.5 min spent, about 98.6 min of steps left; card peak 13.6 GB
|
| 73 |
+
[ 4475.5s] step 1650/4000: task 0.6481 (mse 0.648 cos 0.663); quiet A 0.00102 off 0.00062; 1,022 sites; 2.46 s/step; the job: 74.6 min spent, about 96.5 min of steps left; card peak 13.6 GB
|
| 74 |
+
[ 4598.5s] step 1700/4000: task 0.6367 (mse 0.637 cos 0.668); quiet A 0.00108 off 0.00068; 1,026 sites; 2.46 s/step; the job: 76.6 min spent, about 94.5 min of steps left; card peak 13.6 GB
|
| 75 |
+
[ 4721.5s] step 1750/4000: task 0.6203 (mse 0.620 cos 0.676); quiet A 0.00108 off 0.00065; 993 sites; 2.46 s/step; the job: 78.7 min spent, about 92.4 min of steps left; card peak 13.6 GB
|
| 76 |
+
[ 4844.7s] step 1800/4000: task 0.6067 (mse 0.607 cos 0.683); quiet A 0.00112 off 0.00061; 1,000 sites; 2.46 s/step; the job: 80.7 min spent, about 90.4 min of steps left; card peak 13.6 GB
|
| 77 |
+
[ 4967.9s] step 1850/4000: task 0.5919 (mse 0.592 cos 0.690); quiet A 0.00110 off 0.00066; 999 sites; 2.46 s/step; the job: 82.8 min spent, about 88.3 min of steps left; card peak 13.6 GB
|
| 78 |
+
[ 5090.8s] step 1900/4000: task 0.5794 (mse 0.579 cos 0.695); quiet A 0.00108 off 0.00074; 992 sites; 2.46 s/step; the job: 84.8 min spent, about 86.2 min of steps left; card peak 13.6 GB
|
| 79 |
+
[ 5213.7s] step 1950/4000: task 0.5722 (mse 0.572 cos 0.700); quiet A 0.00116 off 0.00070; 1,030 sites; 2.46 s/step; the job: 86.9 min spent, about 84.2 min of steps left; card peak 13.6 GB
|
| 80 |
+
[ 5336.6s] step 2000/4000: task 0.5601 (mse 0.560 cos 0.706); quiet A 0.00113 off 0.00072; 1,034 sites; 2.46 s/step; the job: 88.9 min spent, about 82.1 min of steps left; card peak 13.6 GB
|
| 81 |
+
[ 5336.8s] anchor saved: /workspace/qarm/out/anchors/sarm_clip_mse_solo_o0_norm_close2_s12_clip.safetensors
|
| 82 |
+
[ 5492.1s] CLOSE 2 (step 2000): the member's alignment read on draw 1 (512 captions, 4,871 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 83 |
+
[ 5492.1s] block 8: 0.478 -> 0.639 / 0.644; A 0.9931; mse 0.574 cos 0.655
|
| 84 |
+
[ 5492.1s] block 12: 0.431 -> 0.722 / 0.731; A 0.9897; mse 0.497 cos 0.730
|
| 85 |
+
[ 5492.1s] block 16: 0.401 -> 0.786 / 0.791; A 0.9886; mse 0.376 cos 0.806
|
| 86 |
+
[ 5492.1s] block 18: 0.390 -> 0.819 / 0.823; A 0.9857; mse 0.324 cos 0.835
|
| 87 |
+
[ 5492.1s] block 20: 0.378 -> 0.843 / 0.847; A 0.9878; mse 0.282 cos 0.859
|
| 88 |
+
[ 5492.1s] block 24: 0.390 -> 0.867 / 0.871; A 0.9853; mse 0.226 cos 0.893
|
| 89 |
+
[ 5492.1s] block 28: 0.316 -> 0.855 / 0.858; A 0.9849; mse 0.224 cos 0.887
|
| 90 |
+
[ 5492.1s] block 31: 0.298 -> 0.829 / 0.832; A 0.9807; mse 0.251 cos 0.859
|
| 91 |
+
[ 5492.1s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.722 / 0.788 / 0.841 / 0.863 / 0.856
|
| 92 |
+
[ 5492.1s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.722 against the bar .85; B2: the lowest A alignment 0.9797 against .995; fineweb share +0.00051 +- 0.00021 against +.012
|
| 93 |
+
[ 5492.1s] gate mean 0.205 / widest 0.283; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 65,441 (skipped {'test': 98, 'spelling': 1054, 'empty': 0, 'short': 0}); card peak 13.6 GB; host RSS 13.0 GB
|
| 94 |
+
[ 5615.2s] step 2050/4000: task 0.5498 (mse 0.550 cos 0.711); quiet A 0.00113 off 0.00071; 1,053 sites; 2.46 s/step; the job: 93.6 min spent, about 80.1 min of steps left; card peak 13.6 GB
|
| 95 |
+
[ 5738.6s] step 2100/4000: task 0.5392 (mse 0.539 cos 0.716); quiet A 0.00120 off 0.00074; 1,019 sites; 2.46 s/step; the job: 95.6 min spent, about 78.0 min of steps left; card peak 13.6 GB
|
| 96 |
+
[ 5861.7s] step 2150/4000: task 0.5307 (mse 0.531 cos 0.720); quiet A 0.00118 off 0.00070; 1,006 sites; 2.46 s/step; the job: 97.7 min spent, about 76.0 min of steps left; card peak 13.6 GB
|
| 97 |
+
[ 5984.8s] step 2200/4000: task 0.5240 (mse 0.524 cos 0.724); quiet A 0.00117 off 0.00068; 1,029 sites; 2.46 s/step; the job: 99.7 min spent, about 73.9 min of steps left; card peak 13.6 GB
|
| 98 |
+
[ 6107.8s] step 2250/4000: task 0.5160 (mse 0.516 cos 0.727); quiet A 0.00116 off 0.00076; 1,013 sites; 2.46 s/step; the job: 101.8 min spent, about 71.9 min of steps left; card peak 13.6 GB
|
| 99 |
+
[ 6230.8s] step 2300/4000: task 0.5123 (mse 0.512 cos 0.729); quiet A 0.00131 off 0.00081; 996 sites; 2.46 s/step; the job: 103.8 min spent, about 69.8 min of steps left; card peak 13.6 GB
|
| 100 |
+
[ 6354.2s] step 2350/4000: task 0.4952 (mse 0.495 cos 0.737); quiet A 0.00116 off 0.00079; 1,046 sites; 2.46 s/step; the job: 105.9 min spent, about 67.7 min of steps left; card peak 13.6 GB
|
| 101 |
+
[ 6477.2s] step 2400/4000: task 0.4929 (mse 0.493 cos 0.739); quiet A 0.00125 off 0.00077; 1,035 sites; 2.46 s/step; the job: 108.0 min spent, about 65.7 min of steps left; card peak 13.6 GB
|
| 102 |
+
[ 6600.2s] step 2450/4000: task 0.4815 (mse 0.482 cos 0.744); quiet A 0.00125 off 0.00085; 1,007 sites; 2.46 s/step; the job: 110.0 min spent, about 63.6 min of steps left; card peak 13.6 GB
|
| 103 |
+
[ 6723.2s] step 2500/4000: task 0.4743 (mse 0.474 cos 0.748); quiet A 0.00117 off 0.00075; 1,034 sites; 2.46 s/step; the job: 112.1 min spent, about 61.6 min of steps left; card peak 13.6 GB
|
| 104 |
+
[ 6846.1s] step 2550/4000: task 0.4689 (mse 0.469 cos 0.750); quiet A 0.00123 off 0.00079; 1,039 sites; 2.46 s/step; the job: 114.1 min spent, about 59.5 min of steps left; card peak 13.6 GB
|
| 105 |
+
[ 6969.0s] step 2600/4000: task 0.4614 (mse 0.461 cos 0.753); quiet A 0.00121 off 0.00074; 983 sites; 2.46 s/step; the job: 116.2 min spent, about 57.5 min of steps left; card peak 13.6 GB
|
| 106 |
+
[ 7092.2s] step 2650/4000: task 0.4522 (mse 0.452 cos 0.758); quiet A 0.00127 off 0.00089; 1,034 sites; 2.46 s/step; the job: 118.2 min spent, about 55.4 min of steps left; card peak 13.6 GB
|
| 107 |
+
[ 7215.2s] step 2700/4000: task 0.4480 (mse 0.448 cos 0.760); quiet A 0.00129 off 0.00081; 1,006 sites; 2.46 s/step; the job: 120.3 min spent, about 53.4 min of steps left; card peak 13.6 GB
|
| 108 |
+
[ 7338.3s] step 2750/4000: task 0.4443 (mse 0.444 cos 0.762); quiet A 0.00119 off 0.00078; 1,007 sites; 2.46 s/step; the job: 122.3 min spent, about 51.3 min of steps left; card peak 13.6 GB
|
| 109 |
+
[ 7461.3s] step 2800/4000: task 0.4313 (mse 0.431 cos 0.768); quiet A 0.00118 off 0.00074; 1,036 sites; 2.46 s/step; the job: 124.4 min spent, about 49.3 min of steps left; card peak 13.6 GB
|
| 110 |
+
[ 7584.3s] step 2850/4000: task 0.4278 (mse 0.428 cos 0.769); quiet A 0.00123 off 0.00076; 1,002 sites; 2.46 s/step; the job: 126.4 min spent, about 47.2 min of steps left; card peak 13.6 GB
|
| 111 |
+
[ 7707.3s] step 2900/4000: task 0.4241 (mse 0.424 cos 0.771); quiet A 0.00126 off 0.00073; 1,032 sites; 2.46 s/step; the job: 128.5 min spent, about 45.2 min of steps left; card peak 13.6 GB
|
| 112 |
+
[ 7830.5s] step 2950/4000: task 0.4185 (mse 0.419 cos 0.774); quiet A 0.00119 off 0.00071; 1,061 sites; 2.46 s/step; the job: 130.5 min spent, about 43.1 min of steps left; card peak 13.6 GB
|
| 113 |
+
[ 7953.6s] step 3000/4000: task 0.4189 (mse 0.419 cos 0.774); quiet A 0.00117 off 0.00076; 997 sites; 2.46 s/step; the job: 132.6 min spent, about 41.0 min of steps left; card peak 13.6 GB
|
| 114 |
+
[ 7953.9s] anchor saved: /workspace/qarm/out/anchors/sarm_clip_mse_solo_o0_norm_close3_s12_clip.safetensors
|
| 115 |
+
[ 8109.3s] CLOSE 3 (step 3000): the member's alignment read on draw 1 (512 captions, 4,871 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 116 |
+
[ 8109.3s] block 8: 0.478 -> 0.656 / 0.664; A 0.9901; mse 0.467 cos 0.722
|
| 117 |
+
[ 8109.3s] block 12: 0.431 -> 0.752 / 0.760; A 0.9925; mse 0.383 cos 0.797
|
| 118 |
+
[ 8109.3s] block 16: 0.401 -> 0.829 / 0.834; A 0.9899; mse 0.271 cos 0.864
|
| 119 |
+
[ 8109.3s] block 18: 0.390 -> 0.859 / 0.863; A 0.9889; mse 0.232 cos 0.885
|
| 120 |
+
[ 8109.3s] block 20: 0.378 -> 0.876 / 0.879; A 0.9895; mse 0.203 cos 0.901
|
| 121 |
+
[ 8109.3s] block 24: 0.390 -> 0.894 / 0.898; A 0.9882; mse 0.162 cos 0.924
|
| 122 |
+
[ 8109.3s] block 28: 0.316 -> 0.887 / 0.890; A 0.9883; mse 0.162 cos 0.920
|
| 123 |
+
[ 8109.3s] block 31: 0.298 -> 0.867 / 0.869; A 0.9834; mse 0.184 cos 0.901
|
| 124 |
+
[ 8109.3s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.754 / 0.832 / 0.876 / 0.895 / 0.887
|
| 125 |
+
[ 8109.3s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.752 against the bar .85; B2: the lowest A alignment 0.9822 against .995; fineweb share +0.00018 +- 0.00019 against +.012
|
| 126 |
+
[ 8109.3s] gate mean 0.288 / widest 0.542; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 97,781 (skipped {'test': 158, 'spelling': 1583, 'empty': 0, 'short': 0}); card peak 13.6 GB; host RSS 13.1 GB
|
| 127 |
+
[ 8232.4s] step 3050/4000: task 0.4164 (mse 0.416 cos 0.776); quiet A 0.00117 off 0.00071; 1,012 sites; 2.46 s/step; the job: 137.2 min spent, about 39.0 min of steps left; card peak 13.6 GB
|
| 128 |
+
[ 8355.5s] step 3100/4000: task 0.4071 (mse 0.407 cos 0.780); quiet A 0.00135 off 0.00106; 1,037 sites; 2.46 s/step; the job: 139.3 min spent, about 36.9 min of steps left; card peak 13.6 GB
|
| 129 |
+
[ 8478.4s] step 3150/4000: task 0.4085 (mse 0.408 cos 0.780); quiet A 0.00125 off 0.00071; 1,022 sites; 2.46 s/step; the job: 141.3 min spent, about 34.9 min of steps left; card peak 13.6 GB
|
| 130 |
+
[ 8601.5s] step 3200/4000: task 0.3976 (mse 0.398 cos 0.784); quiet A 0.00117 off 0.00074; 1,030 sites; 2.46 s/step; the job: 143.4 min spent, about 32.8 min of steps left; card peak 13.6 GB
|
| 131 |
+
[ 8724.7s] step 3250/4000: task 0.3996 (mse 0.400 cos 0.784); quiet A 0.00120 off 0.00075; 1,006 sites; 2.46 s/step; the job: 145.4 min spent, about 30.8 min of steps left; card peak 13.6 GB
|
| 132 |
+
[ 8847.8s] step 3300/4000: task 0.3903 (mse 0.390 cos 0.789); quiet A 0.00121 off 0.00083; 1,050 sites; 2.46 s/step; the job: 147.5 min spent, about 28.7 min of steps left; card peak 13.6 GB
|
| 133 |
+
[ 8970.8s] step 3350/4000: task 0.3804 (mse 0.380 cos 0.794); quiet A 0.00123 off 0.00074; 1,030 sites; 2.46 s/step; the job: 149.5 min spent, about 26.7 min of steps left; card peak 13.6 GB
|
| 134 |
+
[ 9093.9s] step 3400/4000: task 0.3766 (mse 0.377 cos 0.795); quiet A 0.00119 off 0.00069; 1,018 sites; 2.46 s/step; the job: 151.6 min spent, about 24.6 min of steps left; card peak 13.6 GB
|
| 135 |
+
[ 9216.9s] step 3450/4000: task 0.3679 (mse 0.368 cos 0.800); quiet A 0.00124 off 0.00075; 1,016 sites; 2.46 s/step; the job: 153.6 min spent, about 22.6 min of steps left; card peak 13.6 GB
|
| 136 |
+
[ 9339.9s] step 3500/4000: task 0.3606 (mse 0.361 cos 0.803); quiet A 0.00116 off 0.00074; 1,008 sites; 2.46 s/step; the job: 155.7 min spent, about 20.5 min of steps left; card peak 13.6 GB
|
| 137 |
+
[ 9463.2s] step 3550/4000: task 0.3552 (mse 0.355 cos 0.806); quiet A 0.00116 off 0.00065; 1,011 sites; 2.46 s/step; the job: 157.7 min spent, about 18.5 min of steps left; card peak 13.6 GB
|
| 138 |
+
[ 9586.2s] step 3600/4000: task 0.3526 (mse 0.353 cos 0.807); quiet A 0.00120 off 0.00068; 1,022 sites; 2.46 s/step; the job: 159.8 min spent, about 16.4 min of steps left; card peak 13.6 GB
|
| 139 |
+
[ 9709.1s] step 3650/4000: task 0.3468 (mse 0.347 cos 0.809); quiet A 0.00119 off 0.00074; 1,026 sites; 2.46 s/step; the job: 161.8 min spent, about 14.4 min of steps left; card peak 13.6 GB
|
| 140 |
+
[ 9832.2s] step 3700/4000: task 0.3416 (mse 0.342 cos 0.812); quiet A 0.00115 off 0.00070; 1,034 sites; 2.46 s/step; the job: 163.9 min spent, about 12.3 min of steps left; card peak 13.6 GB
|
| 141 |
+
[ 9955.3s] step 3750/4000: task 0.3408 (mse 0.341 cos 0.813); quiet A 0.00120 off 0.00085; 1,025 sites; 2.46 s/step; the job: 165.9 min spent, about 10.3 min of steps left; card peak 13.6 GB
|
| 142 |
+
[10078.4s] step 3800/4000: task 0.3377 (mse 0.338 cos 0.815); quiet A 0.00114 off 0.00066; 1,001 sites; 2.46 s/step; the job: 168.0 min spent, about 8.2 min of steps left; card peak 13.6 GB
|
| 143 |
+
[10201.3s] step 3850/4000: task 0.3275 (mse 0.328 cos 0.819); quiet A 0.00113 off 0.00064; 1,025 sites; 2.46 s/step; the job: 170.0 min spent, about 6.2 min of steps left; card peak 13.6 GB
|
| 144 |
+
[10324.4s] step 3900/4000: task 0.3271 (mse 0.327 cos 0.819); quiet A 0.00113 off 0.00067; 1,020 sites; 2.46 s/step; the job: 172.1 min spent, about 4.1 min of steps left; card peak 13.6 GB
|
| 145 |
+
[10447.5s] step 3950/4000: task 0.3231 (mse 0.323 cos 0.822); quiet A 0.00119 off 0.00083; 1,013 sites; 2.46 s/step; the job: 174.1 min spent, about 2.1 min of steps left; card peak 13.6 GB
|
| 146 |
+
[10570.6s] step 4000/4000: task 0.3218 (mse 0.322 cos 0.822); quiet A 0.00109 off 0.00065; 1,023 sites; 2.46 s/step; the job: 176.2 min spent, about 0.0 min of steps left; card peak 13.6 GB
|
| 147 |
+
[10570.8s] anchor saved: /workspace/qarm/out/anchors/sarm_clip_mse_solo_o0_norm_close4_s12_clip.safetensors
|
| 148 |
+
[10726.1s] CLOSE 4 (step 4000): the member's alignment read on draw 1 (512 captions, 4,871 sites); per block: before -> armed B vs masked A (target) / armed B vs armed A; A armed vs masked
|
| 149 |
+
[10726.1s] block 8: 0.478 -> 0.676 / 0.681; A 0.9928; mse 0.385 cos 0.774
|
| 150 |
+
[10726.1s] block 12: 0.431 -> 0.786 / 0.792; A 0.9942; mse 0.305 cos 0.841
|
| 151 |
+
[10726.1s] block 16: 0.401 -> 0.867 / 0.869; A 0.9904; mse 0.209 cos 0.896
|
| 152 |
+
[10726.1s] block 18: 0.390 -> 0.889 / 0.890; A 0.9903; mse 0.178 cos 0.913
|
| 153 |
+
[10726.1s] block 20: 0.378 -> 0.901 / 0.903; A 0.9913; mse 0.157 cos 0.924
|
| 154 |
+
[10726.1s] block 24: 0.390 -> 0.917 / 0.918; A 0.9889; mse 0.127 cos 0.941
|
| 155 |
+
[10726.1s] block 28: 0.316 -> 0.910 / 0.911; A 0.9886; mse 0.126 cos 0.938
|
| 156 |
+
[10726.1s] block 31: 0.298 -> 0.892 / 0.892; A 0.9843; mse 0.144 cos 0.923
|
| 157 |
+
[10726.1s] draw 2 (the repeat), target alignment at 12/16/20/24/28: 0.787 / 0.871 / 0.901 / 0.917 / 0.911
|
| 158 |
+
[10726.1s] B1 (served blocks >= 12, draw 1): the lowest target alignment 0.786 against the bar .85; B2: the lowest A alignment 0.9814 against .995; fineweb share +0.00019 +- 0.00023 against +.012
|
| 159 |
+
[10726.1s] gate mean 0.369 / widest 0.694; compiled chain: 5 cache entries (limit 256), 0 limit warnings; docs drawn 130,476 (skipped {'test': 206, 'spelling': 2100, 'empty': 0, 'short': 0}); card peak 13.6 GB; host RSS 13.1 GB
|
| 160 |
+
[10726.2s] DONE sarm_clip_mse_solo_o0_norm: 4000 steps in 164.2 min of steps (2.46 s/step); 4 closes; 10726 s in all; card peak 13.6 GB; host RSS 13.1 GB; compiled chain: 5 cache entries (limit 256), 0 limit warnings
|