File size: 10,720 Bytes
860a259
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
component_id,organization,family,low_musd,central_musd,high_musd,documented_input_central_musd,scenario_remainder_central_musd,sources,method,overlap_review,lineage_uncertain,flops_1e24,evidence_grade,unresolved
cosmos1-wfm,NVIDIA,Cosmos 1 / Predict1 β€” allocation scenario,8.1,27.0,64.8,0,27.0,"[""https://arxiv.org/html/2501.03575""]","Reported cluster 10,000 H100 over three months; 90-day approximation. ASSUME 25/50/75% of cluster-capacity hours attributable to retained model training (not utilisation measurement), at $1.5/2.5/4 per GPU-hour. Excludes separately adding constituent models. Allocation is not empirically calibrated.","Entire released WFM group counted once; no separate Transfer1, tokenizer or inherited NeMo/T5 base added.",False,,C,No actual dollar invoice; price and FLOP conversion assumptions. 
cosmos3-generator,NVIDIA,Cosmos 3 Nano + Super β€” generator increment,4.319616,9.719136,43.19616,0,9.719136,"[""https://arxiv.org/html/2606.02800v4""]","Architecture scenario: 6ND with one 8B/32B tower per token Γ— (31.05+2.4)T/(17.86+1.9)T tokens. Token counts reported; effective training compute proxy uncertain. No use of ambiguous benchmark throughput. High doubles compute proxy for architecture/attention overhead. This is a sensitivity assumption, not a measured bound.",Qwen3-VL upstream excluded; Edge/Alpamayo/posttraining excluded. Separate generator training only.,False,"[5.39952, 5.39952, 5.39952]",C,No actual dollar invoice; price and FLOP conversion assumptions. 
command-a,Cohere,Command A β€” 111B base,0,21.5784,79.92,0,21.5784,"[""https://huggingface.co/CohereLabs/c4ai-command-a-03-2025""]","6ND proxy; 111B active Γ— [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Remove legacy Command R/R+ successor allowance first; Vision/Reasoning/Aya not added separately.,True,"[6.66, 11.988, 19.98]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
command-aplus,Cohere,Command A+ β€” 25B active base,0,6.75,24.0,0,6.75,"[""https://huggingface.co/CohereLabs/command-a-plus-05-2026-bf16""]","6ND proxy; 25B active Γ— [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Separate-generation assumption; no Vision/Translate/North increments added.,True,"[2.25, 3.75, 6.0]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
grok2,xAI,Grok 2 β€” 2024 base scenario,0,9.763200000000001,40.68,0,9.763200000000001,"[""https://huggingface.co/xai-org/grok-2""]","6ND proxy; 113B active Γ— [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Grok1 separately counted; active113B is config-derived approximation; independent-run assumption; no additional instruct run.,True,"[2.712, 5.424, 10.17]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
axk1,SK Telecom,A.X K1,1.584,6.4152000000000005,23.76,0,6.4152000000000005,"[""https://huggingface.co/skt/A.X-K1""]","6ND proxy; 33B active Γ— [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Card explicitly says from scratch; A.X3.1 omitted.,False,"[1.98, 3.564, 5.94]",C,No actual dollar invoice; price and FLOP conversion assumptions. 
axk2,SK Telecom,A.X K2 β€” pretraining,1.29888,2.9224799999999997,6.4944,2.9224799999999997,0,"[""https://huggingface.co/skt/A.X-K2""]","6ND proxy; 33B active Γ— [8.2, 8.2, 8.2]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.",Card explicitly says from scratch; K1 not reused as a charged base. Posttraining not counted.,False,"[1.6236, 1.6236, 1.6236]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
qwen2-dense,Alibaba,Qwen 2 β€” dense 0.5/1.5/7/72B,2.7336,6.1506,13.668,6.1506,0,"[""https://arxiv.org/html/2407.10671v1""]","Reported schedule: 0.5BΓ—12T + (1.5+7+72)BΓ—7T; 6ND; prices unchanged. 7T is a conservative reported slice, excludes long-context overhead.",Pre-Qwen2.5 generation; each dense size once; excludes Qwen1/1.5 unresolved overlap.,False,"[3.417, 3.417, 3.417]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
qwen2-moe,Alibaba,Qwen 2 57B-A14B β€” upcycling increment,0.3024,0.6804,1.512,0.6804,0,"[""https://arxiv.org/html/2407.10671v1""]","6ND proxy; 14B active Γ— [4.5, 4.5, 4.5]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.",Only additional4.5T after Qwen2-7B; parent is already in preceding dense entry.,False,"[0.378, 0.378, 0.378]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
glm4-32,Z.ai,GLM 4 32B β€” April 2025 base,2.304,5.184,11.52,5.184,0,"[""https://huggingface.co/zai-org/GLM-4-32B-Base-0414""]","6ND proxy; 32B active Γ— [15, 15, 15]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.",Distinct from GLM4.5 and5; Z1/Rumination/Flash/vision not added.,False,"[2.88, 2.88, 2.88]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
seedoss,ByteDance Seed,Seed-OSS 36B β€” base,2.0736000000000003,4.6656,10.368,4.6656,0,"[""https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Base""]","6ND proxy; 36B active Γ— [12, 12, 12]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","One base only; woSyn alternative, Seed-Coder and BAGEL excluded.",False,"[2.592, 2.592, 2.592]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
chameleon,Meta,Chameleon 7B + 34B β€” pretraining,5.138888,10.277776,15.416664,10.277776,0,"[""https://arxiv.org/html/2405.09818v1""]","5138888 GPU-hours Γ— [1, 2, 3] USD/GPU-hour (assumed prices).",Separate early-fusion runs; no Llama base charged.,False,,B,No actual dollar invoice; price and FLOP conversion assumptions. 
codellama,Meta,Code Llama β€” released 12-model increment,1.4,2.8,4.2,2.8,0,"[""https://huggingface.co/meta-llama/CodeLlama-70b-hf""]","1400000 GPU-hours Γ— [1, 2, 3] USD/GPU-hour (assumed prices).","Llama2 upstream already in V3; all12 includes70B. Older400k report is subset, not extra.",False,,B,No actual dollar invoice; price and FLOP conversion assumptions. 
cwm,Meta,CWM β€” pretraining + mid-training,1.9968000000000001,4.4928,9.984,4.4928,0,"[""https://huggingface.co/facebook/cwm-pretrain""]","6ND proxy; 32B active Γ— [13, 13, 13]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.",8T+5T once across pretrain/SFT/final checkpoints; RL/SFT extra excluded.,False,"[2.496, 2.496, 2.496]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
jamba15,AI21 Labs,Jamba 1.5 Large β€” base scenario,0,8.121599999999999,33.84,0,8.121599999999999,"[""https://arxiv.org/html/2408.12570v1""]","6ND proxy; 94B active Γ— [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",One Large lineage;1.6/1.7/Mini/Jamba2 not separately costed.,True,"[2.256, 4.512, 8.46]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
step3,StepFun,Step 3 β€” base scenario,0,7.3872,27.36,0,7.3872,"[""https://huggingface.co/stepfun-ai/step3""]","6ND proxy; 38B active Γ— [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","V3 Step5 explicitly excluded Step3; distinct-run assumption, vision extra excluded.",True,"[2.28, 4.104, 6.84]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
step35,StepFun,Step 3.5 Flash β€” base scenario,0,2.9699999999999998,10.56,0,2.9699999999999998,"[""https://huggingface.co/stepfun-ai/Step-3.5-Flash-Base""]","6ND proxy; 11B active Γ— [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","Separate base assumed, not proven from architecture.3.7 and midtrain checkpoint not charged separately.",True,"[0.99, 1.65, 2.64]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
olmo3,Ai2,OLMo 3 β€” 7B + 32B pretraining,1.0440479999999999,2.3491079999999998,5.2202399999999995,2.3491079999999998,0,"[""https://huggingface.co/allenai/Olmo-3-1125-32B""]",Reported5.93T/5.50T; nominal7B/32B;6ND. Later stages omitted.,OLMo2 allowance limited to OLMo2; OLMo3 treated as new pretraining generation.,False,"[1.3050599999999999, 1.3050599999999999, 1.3050599999999999]",B,No actual dollar invoice; price and FLOP conversion assumptions. 
mistral-small4,Mistral AI,Mistral Small 4 β€” base scenario,0,1.755,6.24,0,1.755,"[""https://huggingface.co/mistralai/Mistral-Small-4-119B-2603""]","6ND proxy; 6.5B active Γ— [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Independent/upcycled status unknown. Leanstral not added.,True,"[0.585, 0.975, 1.56]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
mistral-small3,Mistral AI,Mistral Small 3 β€” 24B base scenario,0,4.6656,17.28,0,4.6656,"[""https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501""]","6ND proxy; 24B active Γ— [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","One24B lineage;Small3.1/3.2,Magistral,DevstralSmall,VoxtralSmall not separately added.",True,"[1.44, 2.592, 4.32]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.
mistral-nemo,Mistral AI,Mistral NeMo β€” 12B joint base,0.2304,1.0368,4.32,0,1.0368,"[""https://huggingface.co/mistralai/Mistral-Nemo-Base-2407""]","6ND proxy; 12B active Γ— [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Joint NVIDIA/Mistral model counted once under Mistral; NVIDIA mirror and Pixtral not additional base.,False,"[0.288, 0.576, 1.08]",C,No actual dollar invoice; price and FLOP conversion assumptions. 
mistral7,Mistral AI,Mistral 7B β€” original base,0.05184000000000001,0.1944,0.6912,0,0.1944,"[""https://huggingface.co/mistralai/Mistral-7B-v0.1""]","6ND proxy; 7.2B active Γ— [1.5, 2.5, 4]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",v0.2/v0.3/Mathstral not separately added; Mamba excluded.,False,"[0.06480000000000001, 0.108, 0.1728]",C,No actual dollar invoice; price and FLOP conversion assumptions. 
codestral,Mistral AI,Codestral 22B β€” one base scenario,0,1.9180799999999998,7.991999999999999,0,1.9180799999999998,"[""https://huggingface.co/mistralai/Codestral-22B-v0.1""]","6ND proxy; 22.2B active Γ— [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.",Small2409 may share base: not added; Ministral8B2410 excluded.,True,"[0.5327999999999999, 1.0655999999999999, 1.9979999999999998]",C,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.