Spaces:
Running
Running
| organization,family,evidence_grade,low_musd,central_musd,high_musd,allowance_factors,method,sources,v1_central_musd,audit_status,v2_decision,documented_input_central_musd,scenario_remainder_central_musd,audit_fact,unresolved,component_id,overlap_review,lineage_uncertain,flops_1e24 | |
| Meta,Llama 2,A,3.3,7.26,12.87,"[1, 1.1, 1.3]","3.3 millions heures A100; $/h = (1, 2, 3). Famille publiée; supplément de post-training supposé.","[""https://github.com/meta-llama/llama/blob/main/MODEL_CARD.md""]",7.26,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,6.6,0.6600000000000001,,,,,, | |
| Meta,Llama 3 / 3.1 / 3.2 / 3.3,A,63.354,110.86949999999999,249.68,"[1, 1.05, 1.25]",3.1 39.3M H100-h + 3.2 text0.916M + vision2.02M =42.236M. Llama3.0 extra7.7M appears ONLY in high pending overlap resolution; 3.3 repeated7.0M not added.,"[""https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md"", ""https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD.md"", ""https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/MODEL_CARD_VISION.md"", ""https://github.com/meta-llama/llama-models/blob/main/models/llama3/MODEL_CARD.md"", ""https://github.com/meta-llama/llama-models/blob/main/models/llama3_3/MODEL_CARD.md""]",112.9875,Independent Claude source audit + lead review,Replace generic margin by reported3.2 hours; unresolved3.0 overlap not added centrally.,105.59,5.2794999999999845,"3.1 card: 8B 1.46M, 70B 7.0M, 405B 30.84M = 39.3M H100-h (verified). 3.2 card: 916k cumulative (1B 370k, 3B 460k, +86k logit generation). 3.2-Vision card: 2.02M (11B 147k+98k, 90B 885k+885k; SFT/RLHF <6k). Llama 3 (Apr 2024) card: 7.7M (8B 1.3M, 70B 6.4M). 3.3 card repeats 39.3M cumulative with a 7.0M 70B row.","Whether 3.1 8B/70B were continued from 3.0 checkpoints (if yes, 3.0 hours are still real spend for separately released weights but drop for the ""count base once"" rule). 3.2 card text 916k vs table total 833k+86k.",,,, | |
| Meta,Llama 4 Scout + Maverick,A,11.07,20.295,38.376,"[1, 1.1, 1.3]","7.38 millions heures H100; $/h = (1.5, 2.5, 4). Scout 5M h; Maverick 2.38M h. Behemoth non publié exclu.","[""https://github.com/meta-llama/llama-models/blob/main/models/llama4/MODEL_CARD.md""]",20.295,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,18.45,1.8450000000000024,"Card: ""Model pre-training utilized a cumulative of 7.38M GPU hours"" H100-80GB; Scout ~40T, Maverick ~22T tokens. Hours are PRE-training only.",Post-training/RL hours; co-distillation teacher cost.,,,, | |
| Alibaba,Qwen 2.5,B,7.5168,30.44304,90.2016,"[1, 1.2, 1.6]","C (10^24 FLOP), bas/central/haut = [9.396, 14.094, 14.094]. Somme tailles 0.5+1.5+3+7+14+32+72=130.5B; jusqu’à 18T tokens. 12T borne basse car corpus ne garantit pas consommation égale par taille. Coder et VL : marge 0/20/60%, non mesurée.","[""https://huggingface.co/Qwen/Qwen2.5-72B"", ""https://qwenlm.github.io/blog/qwen2.5/""]",30.44304,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,0,30.44304,"Blog/report: pretraining scaled to ""up to 18T tokens""; Epoch applies 18T to every size (e.g. 72B 7.8e24).",Per-size token counts not published; Coder (+5.5T continued) and VL in margin.,,,, | |
| Alibaba,Qwen 3,B,13.76,41.96,129.73,"[1, 1.15, 1.6]","C=1e24×[17.2, 20.27, 20.27]: 85.3B active-sum × 30/36/36T (report) + Qwen3-Coder-480B-A35B 1.575 + Qwen3-Next-80B-A3B 0.27 (Epoch 6ND, 7.5T/15T). Coder7.5T independently confirmed in official Qwen blog; Next15T in official card (not only Epoch).","[""https://huggingface.co/Qwen/Qwen3-235B-A22B"", ""https://arxiv.org/html/2505.09388v1"", ""https://huggingface.co/Qwen/Qwen3-Coder-480B-A35B-Instruct"", ""https://qwenlm.github.io/blog/qwen3-coder/"", ""https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct""]",39.797568,Independent Claude source audit + lead review,Accepted documented quantity corrections; unreported components stay explicitly assumed.,30.958199999999998,11.001800000000003,"Report: ""all Qwen3 models are trained on over 30 trillion tokens"" (S1) + ~5T S2 + long-context S3; corpus 36T. Small models get strong-to-weak distillation only in POST-training, pretraining is full. Coder-480B card: 480B/35B activated, own pretraining stage.",Qwen3-Coder 7.5T and Qwen3-Next 15T taken from Epoch (secondary); Qwen3-2507 re-post-training and VL/Omni increments unmeasured.,,,, | |
| Alibaba,Qwen 3.5,C,5.2416,28.30464,178.2144,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [6.551999999999999, 13.103999999999997, 26.207999999999995]. Tailles actives 17+10+3+27+9+4+2+0.8B. Tokens NON publiés : hypothèse 15/30/60T par taille; indépendance entre tailles supposée. V1 token prior15/30/60T deliberately unchanged; moving it to20/36/60T is judgement, not new data.","[""https://huggingface.co/Qwen/Qwen3.5-397B-A17B"", ""https://qwen.ai/blog?id=qwen3.5""]",28.30464,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,0,28.30464,"Active sizes verified from cards/configs (397B-A17B, 122B-A10B, 35B-A3B, 27B, 9B, 4B, 2B, 0.8B = 72.8B). Separate *-Base repos published for 0.8B/2B/4B/9B/35B-A3B → each size has its own pretraining (independence plausible). No token count in cards; blog JS-only; secondary reports say only ""trillions""/""larger visual-text tokens than Qwen3"". Qwen3.6-27B/35B-A3B configs identical to 3.5 → 3.6 is continued/post-training, not new bases.",Token counts per size; whether small dense sizes were pruned/distilled from larger ones.,,,, | |
| Alibaba,Qwen 3.8,C,6.84,38.475,273.6,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [8.55, 17.1, 34.2]. 95B actifs; tokens NON publiés : 15/30/60T. Hypothèse nouvel entraînement; filiation avec 3.5 à confirmer. Flash-Next is an unpriced candidate; architecture differences alone do not prove from-scratch initialization.","[""https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B"", ""https://huggingface.co/Qwen/Qwen3.8-Flash-Next"", ""https://huggingface.co/Qwen/Qwen3.8-27B""]",38.475,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,0,38.475,"HF card: 2.4T/95B activated, ""Training Stage: Pre-training & Post-training"", ""Built on the architectural foundation of Qwen3.5"" (architecture, not weights). Config hidden 8192/92 layers vs Qwen3.5-397B 4096/60 → cannot be a continuation of 3.5 weights: new base. Qwen3.8-27B config identical to Qwen3.5-27B and Qwen3.6-27B → likely continued training of same base. Qwen3.8-Flash-Next (new arch Qwen4Exp, 125B total/6B active + 51B n-gram embeddings) is a separate new base absent from V1. No token count found in card, web or press (NVIDIA/AWS deployment blogs).",Tokens for both bases. Whether Flash-Next was distilled/initialised from the 2.4T model.,,,, | |
| DeepSeek,DeepSeek V2,B,0.81648,2.112642,6.1236,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [1.0206, 1.0206, 1.0206]. 21B actifs × 8.1T tokens; Lite et adaptations via marge.","[""https://huggingface.co/deepseek-ai/DeepSeek-V2""]",2.112642,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,1.83708,0.2755620000000001,,,,,, | |
| DeepSeek,DeepSeek V3 / R1 / V3.x,A,2.788,8.364,25.092,"[1, 1.5, 3]","2.788 millions heures H800; $/h = (1, 2, 3). V3 publié = 5.576 M$ à 2$/h, dont 2.664M h prétraining. R1 et suites NON couverts par ce chiffre : ajout hypothétique 0/50/200%.","[""https://huggingface.co/deepseek-ai/DeepSeek-V3-Base"", ""https://arxiv.org/html/2412.19437v2""]",8.364,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,5.576,2.788000000000001,,,,,, | |
| DeepSeek,DeepSeek V4 Pro + Flash,B,9.7584,26.34768,82.9464,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [12.198, 12.198, 12.198]. Pro 49B actifs×33T + Flash 13B×32T. 6ND approximatif pour architecture hybride.","[""https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro"", ""https://arxiv.org/html/2606.19348""]",26.34768,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,21.9564,4.391280000000002,"Report: ""we train DeepSeek-V4-Flash on 32T tokens and DeepSeek-V4-Pro on 33T""; card: 1.6T/49B and 284B/13B activated. No GPU-hours published (unlike V3). V4-Flash-0731 is re-post-training only (DeepSeek API changelog via Epoch). V4.1-Flash is a separate from-scratch 45T run (separate V1 family; regex excludes it correctly).",Sparse-attention/hybrid architecture effect on 6ND; Ascend vs NVIDIA share of training hardware.,,,, | |
| DeepSeek,DeepSeek V4.1 Flash,C,1.728,9.72,69.12,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [2.16, 4.32, 8.64]. 45T depuis zéro; 8B actifs prefill / 16B decode. Coût effectif entraînement non donné : 8/16/32B équivalents, pas une mesure.","[""https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash"", ""https://arxiv.org/html/2609.19969""]",9.72,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,9.72,,,,,, | |
| Moonshot,Kimi K2 / K2.5,B,4.6848,12.64896,39.8208,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [5.856, 5.856, 5.856]. 32B actifs; K2 15.5T + K2.5 15T supplémentaires. Base K2 comptée une fois.","[""https://huggingface.co/moonshotai/Kimi-K2-Base"", ""https://arxiv.org/html/2602.02276""]",12.64896,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,10.5408,2.10816,,,,,, | |
| Moonshot,Kimi K3,C,7.488,42.12,299.52,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [9.36, 18.72, 37.44]. 104B actifs. Tokens non publiés; hypothèse 15/30/60T, centrale cohérente avec inférence Epoch, non confirmation.","[""https://huggingface.co/moonshotai/Kimi-K3"", ""https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf""]",42.12,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,0,42.12,"Report (local PDF→text, 190 kB) gives 2.8T total/104B active, 896 experts (16 active), KDA+AttnRes+NoPE, cosine LR, 8K→64K pretrain then 256K→1M cooldown, ~2.5× scaling efficiency vs K2, RL on ""a few hundred GPUs"" per 1M-context experiment, 51.2M sandboxes. NO token count, GPU count, GPU-hours or FLOP total anywhere in the report. HF config: 93 layers, 896 experts vs K2.x 61 layers/384 experts → new base, no K2 double count.",Pretraining tokens; multimodal (from-scratch MoonViT-V2) token share; RL compute for 9 expert models + multi-teacher consolidation.,,,, | |
| Meituan,LongCat Flash + 2.0,B,10.0,29.82096,204.0,"[1, 1.2, 1.5]","C (10^24 FLOP), bas/central/haut = [12.5, 13.806, 17]. Flash ~27B actifs×23T (23T estimé Epoch); 2.0 48B×>35T. Mention « millions accelerator-days » sans prix ASIC : borne haute élargie.","[""https://huggingface.co/meituan-longcat/LongCat-2.0"", ""https://arxiv.org/html/2509.01322""]",29.82096,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,23.976000000000003,5.844959999999997,"Flash paper: 27B average activated (18.6–31.3B), ""more than 20 trillion tokens within 30 days"", ""tens of thousands of accelerators"" (Epoch 23T includes mid-training ""trillions""). LongCat-2.0 card: 1.6T/~48B activated, "">35 trillion tokens"", ""millions of accelerator-days"" on AI-ASIC superpods.","ASIC type and rental-equivalent price; exact token count beyond "">35T"".",,,, | |
| Ant Group,Ling 1T / 2.6,B,7.104,18.3816,53.28,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [8.88, 8.88, 8.88]. Base 50B actifs×20T; Ling2.6 ajoute 9.6T après migration. Ne pas compter deux bases 20T.","[""https://huggingface.co/inclusionAI/Ling-1T"", ""https://huggingface.co/inclusionAI/Ling-2.6-1T-base""]",18.3816,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,15.984000000000002,2.397599999999997,,,,,, | |
| Z.ai,GLM 4.5 / Air,B,4.87872,13.172544,41.46912,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [6.098400000000001, 6.098400000000001, 6.098400000000001]. 32B+12B actifs; 23.1T; variantes officialisées dans marge.","[""https://huggingface.co/zai-org/GLM-4.5-Base""]",13.172544,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,10.97712,2.195424000000001,,,,,, | |
| Z.ai,GLM 5 / 5.x,B,5.472,15.39,54.72,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [6.84, 6.84, 6.84]. 40B actifs×28.5T; prix FLOP équivalent GPU malgré matériel Ascend. Suites et RL ajout 0/25/100%.","[""https://huggingface.co/zai-org/GLM-5""]",15.39,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,12.312,3.078000000000001,,,,,, | |
| Tencent,Hunyuan Large / Hy3 / Hy4,C,5.144,23.4144,169.04,"[1, 1.2, 2]","C (10^24 FLOP), bas/central/haut = [6.43, 10.84, 21.130000000000003]. Hunyuan Large C≈3.49e24 selon Epoch; Hy4 49B actifs. Tokens Hy4 supposés 10/25/60T, Hy3 non ajouté comme nouvelle base faute de filiation; risque de double compte Large/Hy4 restant. CORRECTION TO SCOPE: this monetary scenario covers Large + Hy4 only; Hy3 has no separate priced run and is listed as an unpriced candidate. Reported9.59ND FLOPs retained over generic6ND.","[""https://huggingface.co/tencent/Hy4-preview"", ""https://epoch.ai/data/ai-models"", ""https://arxiv.org/html/2411.02265"", ""https://huggingface.co/tencent/Hy3-preview-Base"", ""https://huggingface.co/tencent/Hy3""]",23.4144,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,6.282000000000001,17.1324,"Hunyuan-Large paper: 389B/52B, ""pre-trained on 7T tokens"", ~1.5T synthetic, compute formula C≈9.59ND+2.3e8D (Epoch 3.49e24 = 9.59×52e9×7e12). Hy3-preview(-Base) card/config: 295B/21B, 80 layers, 192 experts, published Base checkpoint (Apr 2026); Hy3 (Jul 2026) same config → continuation. Hy4-preview: 770B/49B, 78 layers, hidden 6144, 256 experts, Gated DSA → different architecture, new base. No token counts for Hy3/Hy4 anywhere found.","Hy3/Hy4 tokens; whether Hy4 initialised from Hy3 (impossible weight-wise without growth, but not documented).",,,, | |
| LG AI Research,K-EXAONE 1 / 2,C,2.84,10.9512,34.476,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [3.55, 5.07, 5.07]. Estimates Epoch : K2 3.55e24, K1 1.52e24. Faute de filiation démontrée, bas suppose coût K1 inclus, central/haut indépendant; EXAONE antérieurs dans résiduel.","[""https://huggingface.co/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B"", ""https://epoch.ai/data/ai-models""]",10.9512,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,10.9512,,,,,, | |
| NVIDIA,Nemotron 3 / 3.5,B,7.55,21.25,75.55,"[1, 1.25, 2]",C=1e24×[9.44]: Ultra 55B×20T + Super 12B×25T + Nano 3.2B×25T + 3.5 Lightning 3B×20T + 3.5 Super-VL increment (1T distillation from Ultra + ~0.3T CPT; student+teacher-forward ≈0.20).,"[""https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16"", ""https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Base-BF16"", ""https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16"", ""https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-Base-BF16"", ""https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16""]",18.5193,Independent Claude source audit + lead review,Accepted documented quantity corrections; unreported components stay explicitly assumed.,15.984,5.266,"Ultra card: 550B/55B, ""pre-trained with around 20T tokens"", NVFP4. Super card: base ""pre-trained for over 25T tokens"", NVFP4. Nano card: ""trained from scratch"", ""approximately 25 trillion tokens"" (the 10.6T in V1 is the unique-token dataset table, not tokens trained); Epoch: 3.2B active. Lightning card: 30B/3B, "">20T tokens"", NVFP4, data cutoff Sep 2025 (≠ Nano Jun 2025) → separate run. 3.5 Super-VL card: initialised from 3 Super Base, 100B MTP + 1T distillation from Ultra Base + 100B Indic + 35B long-ctx + 67B SWE.",NVFP4 pretraining: FLOP→$ at BF16/FP8-calibrated prices may overstate cost by up to ~2× for Ultra/Super/Lightning (Blackwell FP4 throughput); low scenario does not fully capture this.,,,, | |
| MiniMax,MiniMax Text-01 / M1,B,4.694048,10.836018999999999,29.678336,"[1, 1.1, 1.4]","C (10^24 FLOP), bas/central/haut = [5.20506, 5.20506, 5.20506]. Text01 7.2+3.2+1=11.4T; M1 continuation 7.5T. RL publié ~$0.53M ajouté ci-dessous; pas double base. + 0.53 M$ RL dans chaque scénario.","[""https://huggingface.co/MiniMaxAI/MiniMax-Text-01"", ""https://arxiv.org/html/2506.13585"", ""https://arxiv.org/html/2501.08313""]",10.836018999999999,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,9.899108,0.9369109999999985,,,,,, | |
| MiniMax,MiniMax M2 / M3,C,1.584,11.1375,79.2,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [1.98, 4.95, 9.9]. 10B/23B actifs. Tokens 10/25/50T et bases distinctes supposés. Rapport M3 lié décrit expérience109B/3T : ce chiffre n’est PAS retenu pour M3.","[""https://huggingface.co/MiniMaxAI/MiniMax-M2"", ""https://huggingface.co/MiniMaxAI/MiniMax-M3""]",11.1375,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,11.1375,,,,,, | |
| Baidu,ERNIE 4.5 text / VL,C,2.256,7.56,40.8,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [2.82, 3.5, 6]. 47B actifs; 10T hypothèse Epoch pour texte=2.82e24. VL partage étapes texte : enveloppe plutôt que seconde base entière.","[""https://huggingface.co/baidu/ERNIE-4.5-VL-424B-A47B-Base-PT"", ""https://epoch.ai/data/ai-models""]",7.56,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,7.56,,,,,, | |
| StepFun,Step 5,C,1.296,9.1125,64.8,"[1, 1.25, 2]","C (10^24 FLOP), bas/central/haut = [1.62, 4.05, 8.1]. 27B actifs; tokens non connus 10/25/50T. Step3 et autres familles en résiduel.","[""https://huggingface.co/stepfun-ai/Step-5-Preview""]",9.1125,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,9.1125,,,,,, | |
| Mistral AI,Mixtral 8x7B / 8x22B,C,0.74736,6.44598,37.368,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [0.9341999999999999, 3.114, 6.228]. 12.9/39B actifs; tokens 3/10/20T supposés. Central10T reprend hypothèse Epoch, non chiffre publié.","[""https://huggingface.co/mistralai/Mixtral-8x22B-v0.1"", ""https://epoch.ai/data/ai-models""]",6.44598,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,6.44598,,,,,, | |
| Mistral AI,Mistral Medium 3.5,C,3.072,24.8832,156.672,"[1, 1.2, 1.7]","Cumulative shared-backbone scenario, not three independent bases:128B-equivalent ×5/15/30T assumed. Shared ancestry with Devstral2/Large2411 supported by sampled weights; no amount of training inferred from correlation. Original numerical prior retained pending real budget disclosure.","[""https://huggingface.co/mistralai/Mistral-Medium-3.5-128B"", ""https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512"", ""https://mistral.ai/news/vibe-remote-agents-mistral-medium-3-5"", ""https://x.com/mertunsal2020/status/2049551864556143094"", ""https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512"", ""https://huggingface.co/mistralai/Mistral-Large-Instruct-2411""]",24.8832,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,0,24.8832,"HF configs: Medium-3.5 text backbone hidden 12288, 88 layers, FFN 28672, 96 heads/8 KV, vocab 131072, 256K ctx — IDENTICAL to Devstral-2-123B-Instruct-2512 and to Mistral-Large-2411 except vocab (32768). Card: ""our first flagship merged model"" (merges instruct/reasoning/coding), replaces Medium 3.1, Magistral and Devstral 2. Mistral post: ""We trained the vision encoder from scratch"". Mistral researcher (via Epoch) replied ""nowhere close to those flops"" to a 2.3e25 suggestion.",Sampled weights show r0.994–0.999 with Devstral2; strong similarity but direction/common trained checkpoint versus common initialization not proven. Tokens and incremental budget remain unknown.,,,, | |
| TII,Falcon 7B / 40B / 180B,B,6.0921,14.01183,36.5526,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [4.0614, 4.0614, 4.0614]. 180B 43,500 PF-days publiés; 40B1T et7B1.5T. Prix plus élevé pour génération A100.","[""https://arxiv.org/html/2311.16867"", ""https://huggingface.co/tiiuae/falcon-180B""]",14.01183,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,12.1842,1.8276299999999992,,,,,, | |
| BigScience,BLOOM,A,1.078272,2.264371,3.881779,"[1, 1.05, 1.2]","1.078272 millions heures A100; $/h = (1, 2, 3). Heures run final reprises dans Epoch. Fourchette publique 2–5M$ inclut expériences : pas une facture du run final.","[""https://huggingface.co/bigscience/bloom"", ""https://epoch.ai/data/ai-models""]",2.264371,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,2.264371,,,,,, | |
| Black Forest Labs,FLUX 1 / 2,C,1,8,40,"[1, 1, 1]",Budget inconnu. Scénario famille 0.5/3.2/10 millions heures H100 équivalentes ×2/2.5/4$/h. Ni heures ni dollars publiés. Ne comprend pas toutes les générations fermées des teachers.,"[""https://bfl.ai/blog/flux-2"", ""https://huggingface.co/black-forest-labs/FLUX.2-dev""]",8,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,8,,,,,, | |
| Stability AI,Stable Diffusion 1 / 2 / XL / 3 / 3.5,C,1,5,25,"[1, 1, 1]","Scénario cumulé 0.5/2/6.25 millions heures GPU équivalentes ×2/2.5/4$/h; sparse budgets historiques, surtout inconnus pour3.5. Nouvelles versions comptées avec enveloppe, pas par repo.","[""https://arxiv.org/html/2403.03206"", ""https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0""]",5,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,5,,,,,, | |
| Alibaba,Wan 2.1 / 2.2,C,0.5,3,15,"[1, 1, 1]",Scénario 0.25/1.2/3.75 millions heures GPU ×2/2.5/4$/h. Paramètres et corpus publiés insuffisants pour durée; 6ND texte non appliqué aux images.,"[""https://arxiv.org/html/2503.20314"", ""https://huggingface.co/Wan-AI/Wan2.1-T2V-14B""]",3,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,3,,,,,, | |
| OpenAI,Whisper,C,0.12,0.828,4.8,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [0.15, 0.4, 0.8]. Epoch donne v3≈2.7e23 FLOP et v2≈1.1e23; enveloppe versions et incertitude. Pas toutes les adaptations communautaires.","[""https://huggingface.co/openai/whisper-large-v3"", ""https://epoch.ai/data/ai-models""]",0.828,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,0.828,,,,,, | |
| Google,Gemma 2,B,2.0496,5.53392,16.3968,"[1, 1.2, 1.6]","C (10^24 FLOP), bas/central/haut = [2.562, 2.562, 2.562]. 27B13T +9B8T +2B2T; teacher inference/distillation via supplément 0/20/60%.","[""https://arxiv.org/html/2408.00118""]",5.53392,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,4.6116,0.92232,,,,,, | |
| Google,Gemma 3,B,2.592,6.9984,20.736,"[1, 1.2, 1.6]","C (10^24 FLOP), bas/central/haut = [3.24, 3.24, 3.24]. 27B14T +12B12T +4B4T +1B2T. Distillation et suites dans marge. Gemma3n non estimé séparément.","[""https://arxiv.org/html/2503.19786""]",6.9984,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,5.832000000000001,1.1663999999999994,,,,,, | |
| Google,Gemma 4,C,1.8048,9.1368,57.528,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [2.256, 4.23, 8.46]. Tailles31B,26B-A4B,12B; tokens NON publiés. Scénario8/15/30T par taille. TPU convertis en équivalent coût FLOP; ni nombre de TPU ni durée déduite abusivement.","[""https://arxiv.org/html/2607.02770"", ""https://huggingface.co/google/gemma-4-31B""]",9.1368,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,9.1368,,,,,, | |
| OpenAI,gpt-oss 120B / 20B,A,3.465,5.775,12.012,"[1, 1, 1.3]","2.31 millions heures H100; $/h = (1.5, 2.5, 4). Rapport :120B 2.1M h,20B presque10x moins; approximation0.21M h. Posttraining couverture incertaine : pas supplément central pour éviter de recompter.","[""https://arxiv.org/html/2508.10925""]",5.775,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,5.775,0.0,,,,,, | |
| Ai2,OLMo 2,B,1.368,3.5397,10.26,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [1.71, 1.71, 1.71]. 7B4T,13B5T,32B6T; annealing et adaptations dans marge.","[""https://huggingface.co/allenai/OLMo-2-0325-32B""]",3.5397,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,3.078,0.4617,,,,,, | |
| IBM,Granite 3 dense,B,0.576,1.4904,4.32,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [0.72, 0.72, 0.72]. 8B et2B ×12T; MoE, autres générations Granite en résiduel.","[""https://huggingface.co/ibm-granite/granite-3.0-8b-base""]",1.4904,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,1.296,0.1943999999999999,,,,,, | |
| Microsoft,Phi 4,B,0.65856,1.778112,5.59776,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [0.8232, 0.8232, 0.8232]. 14B×9.8T; petites variantes et adaptations dans enveloppe.","[""https://huggingface.co/microsoft/phi-4""]",1.778112,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,1.4817600000000002,0.2963519999999997,,,,,, | |
| NVIDIA,Nemotron 4 / H,B,20.64,53.41,154.8,"[1, 1.15, 1.5]","C=1e24×[25.80]: 340B×9T (8T+1T continued, card) + H-56B×20T + H-8B×15T (paper §2.4). H-47B (63B-token distillation) excluded.","[""https://huggingface.co/nvidia/Nemotron-4-340B-Base"", ""https://arxiv.org/abs/2504.03624""]",51.9156,Independent Claude source audit + lead review,Accepted documented quantity corrections; unreported components stay explicitly assumed.,46.440000000000005,6.969999999999992,"Nemotron-4-340B-Base card: 8T pretrain + 1T continued = 9T. Nemotron-H paper: 8B on 15T, 56B on 20T, 6144 H100, FP8. Local snapshot nvidia--Nemotron-H-56B-Base.txt is a 29-byte failed fetch.",No GPU-hours published for either. FP8 throughput on H100 means FLOP→$ conversion may overstate cost slightly.,,,, | |
| Arcee / Prime Intellect,Trinity Large,B,1.0608,2.74482,7.956,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [1.326, 1.326, 1.326]. 13B actifs×17T finaux; TrueBase10T intermédiaire non ajouté.","[""https://arxiv.org/html/2602.17004"", ""https://huggingface.co/arcee-ai/Trinity-Large-TrueBase""]",2.74482,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,2.3868,0.3580199999999998,,,,,, | |
| StepFun,Step Video,C,1.2,8.490272,48.0,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [1.5, 4.1015808, 8]. Rapport : milliers H800 pendant >1 mois, utilisation32%. Epoch suppose5000 GPU×720h; nombre et durée centraux inférés, non publiés. Texte encodeur non valorisé séparément.","[""https://arxiv.org/html/2502.10248"", ""https://huggingface.co/stepfun-ai/stepvideo-t2v""]",8.490272,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,8.490272,,,,,, | |
| Thinking Machines,Inkling / Small,C,9.72,32.2,123.12,"[1, 1.25, 2]","C=1e24×[12.15, 14.31, 15.39]: Inkling 41B×45T (PRIMARY, fixed) + Small 12B×15/45/60T (prior).","[""https://huggingface.co/thinkingmachines/Inkling"", ""https://huggingface.co/thinkingmachines/Inkling-Small"", ""https://epoch.ai/data/ai-models"", ""https://thinkingmachines.ai/news/introducing-inkling/""]",32.1975,Independent Claude source audit + lead review,Accepted documented quantity corrections; unreported components stay explicitly assumed.,19.926000000000002,12.274000000000001,"TML launch post: Inkling ""pretrained on 45 trillion tokens of text, images, audio and video"", GB300 NVL72, RL ""over 30M rollouts"" in two long runs, SFT bootstrap ""small fraction of compute"". Small: ""trained with a similar recipe"" with data/recipe improvements; no token count, no distillation statement. Cards: 975B/41B and 276B/12B, Apache-2.0, not gated.",Small tokens; whether 45T counts multimodal tokens the same way (video tokens are cheap per token in some encoders); RL compute.,,,, | |
| Xiaomi,MiMo 7B / V2 Flash / V2.5 Pro,B,8.2272,22.21344,69.9312,"[1, 1.2, 1.7]","C (10^24 FLOP), bas/central/haut = [10.284, 10.284, 10.284]. 7B25T; Flash15B27T; Pro42B27T. Autres versions dans marge, pas duplication VL7B.","[""https://huggingface.co/XiaomiMiMo/MiMo-V2-Flash"", ""https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro"", ""https://arxiv.org/abs/2505.07608""]",22.21344,Independent Claude source audit + lead review,Source notes accepted. Priors held stable: architecture alone does not establish weight ancestry; no new evidence for changed token priors.,18.511200000000002,3.702239999999996,"Cards: V2-Flash 309B/15B ""Trained on 27T tokens using FP8""; V2.5-Pro 1.02T/42B ""Trained on 27T tokens using FP8""; MiMo-7B 25T (arXiv 2505.07608, via Epoch).","Whether V2.5-Pro reused V2-Pro (closed) pretraining — if V2-Pro and V2.5-Pro share one 27T run, fine; if V2.5-Pro added more tokens, undercount.",,,, | |
| China Telecom,TeleChat 1 / 2,B,5.62272,15.84,60.0,"[1, 1.1, 1.5]","C (10^24 FLOP), bas/central/haut = [7.0284, 8, 10]. 115B10T explicite=6.9e24; Telechat1 7B+12B=.1284e24. PetitsTelechat2 non chiffrés : enveloppe centrale8e24, haute10e24.","[""https://huggingface.co/Tele-AI/TeleChat2-115B"", ""https://github.com/Tele-AI/TeleChat2""]",15.84,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,12.420000000000002,3.419999999999998,,,,,, | |
| Swiss AI,Apertus 8B / 70B,B,5.968,15.4422,44.76,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [7.46, 7.46, 7.46]. 70B6.74e24 FLOP rapportés selon Epoch;8B15T=.72e24. Équivalent GPU pour GH200.","[""https://huggingface.co/swiss-ai/Apertus-70B-2509"", ""https://github.com/swiss-ai/apertus-tech-report/blob/main/Apertus_Tech_Report.pdf""]",15.4422,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,11.34,4.1022,,,,,, | |
| Upstage,Solar Open 100B / Open2 250B,B,7.13472,12.55312,21.6736,"[1, 1, 1]",Open2 fiche officielle2M heures B200 ×3/5/8$/h. Open100B12B×19.7T=1.4184e24. Pas ajout des coûts32M/8.4M cités par Epoch (périmètre divergent).,"[""https://huggingface.co/upstage/Solar-Open2-250B"", ""https://huggingface.co/upstage/Solar-Open-100B"", ""https://epoch.ai/data/ai-models""]",12.55312,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,12.55312,0.0,,,,,, | |
| Databricks,DBRX,B,2.0736,5.36544,15.552,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [2.592, 2.592, 2.592]. 36B actifs×12T; ne transforme pas les3 mois développement en3 mois de cluster.","[""https://www.databricks.com/blog/introducing-dbrx-new-state-art-open-llm"", ""https://huggingface.co/databricks/dbrx-base""]",5.36544,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,4.6656,0.69984,,,,,, | |
| xAI,Grok 1,C,1.2,6.003,42.0,"[1, 1.15, 1.5]","C (10^24 FLOP), bas/central/haut = [1.5, 2.9, 7]. Tokens non publiés; C central et bornes issus raisonnement comparatif Epoch, pas mesure de dépense.","[""https://huggingface.co/xai-org/grok-1"", ""https://epoch.ai/data/ai-models""]",6.003,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,6.003,,,,,, | |
| Cohere,Command R / R+,C,1.3344,9.007200000000001,50.04,"[1, 1, 1]","C (10^24 FLOP), bas/central/haut = [1.668, 5.004, 12.51]. 35B et104B; tokens inconnus, hypothèse2/6/15T. Suites inconnues dans marge. V4: successor allowance removed before adding Command A/A+.","[""https://huggingface.co/CohereLabs/c4ai-command-r-plus""]",10.80864,V1 source review; not independently re-audited this turn,Retained V1 scenario; not a newly observed budget.,0,9.007200000000001,,,,,, | |
| Mistral AI,Mistral Large 3 — base,,1.968,7.9704,29.52,,"6ND proxy: 41B × [10, 18, 30]T; tokens assumed; active params and from-scratch training verified","[""https://huggingface.co/mistralai/Mistral-Large-3-675B-Base-2512""]",,,,0,7.9704,,Additional stages and actual GPU rental prices unknown.,,,, | |
| NVIDIA,Nemotron Nano 2 — 12B base,,1.1817600000000001,2.65896,5.9088,,"6ND proxy: 12.31B × [20, 20, 20]T; tokens reported; prices assumed","[""https://huggingface.co/nvidia/NVIDIA-Nemotron-Nano-12B-v2-Base""]",,,,2.65896,0,,Additional stages and actual GPU rental prices unknown.,,,, | |
| NVIDIA,Cosmos 1 / Predict1 — allocation scenario,C,8.1,27.0,64.8,,"Reported cluster 10,000 H100 over three months; 90-day approximation. ASSUME 25/50/75% of cluster-capacity hours attributable to retained model training (not utilisation measurement), at $1.5/2.5/4 per GPU-hour. Excludes separately adding constituent models. Allocation is not empirically calibrated.","[""https://arxiv.org/html/2501.03575""]",,,,0,27.0,,No actual dollar invoice; price and FLOP conversion assumptions. ,cosmos1-wfm,"Entire released WFM group counted once; no separate Transfer1, tokenizer or inherited NeMo/T5 base added.",False, | |
| NVIDIA,Cosmos 3 Nano + Super — generator increment,C,4.319616,9.719136,43.19616,,"Architecture scenario: 6ND with one 8B/32B tower per token × (31.05+2.4)T/(17.86+1.9)T tokens. Token counts reported; effective training compute proxy uncertain. No use of ambiguous benchmark throughput. High doubles compute proxy for architecture/attention overhead. This is a sensitivity assumption, not a measured bound.","[""https://arxiv.org/html/2606.02800v4""]",,,,0,9.719136,,No actual dollar invoice; price and FLOP conversion assumptions. ,cosmos3-generator,Qwen3-VL upstream excluded; Edge/Alpamayo/posttraining excluded. Separate generator training only.,False,"[5.39952, 5.39952, 5.39952]" | |
| Cohere,Command A — 111B base,C,0,21.5784,79.92,,"6ND proxy; 111B active × [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/CohereLabs/c4ai-command-a-03-2025""]",,,,0,21.5784,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,command-a,Remove legacy Command R/R+ successor allowance first; Vision/Reasoning/Aya not added separately.,True,"[6.66, 11.988, 19.98]" | |
| Cohere,Command A+ — 25B active base,C,0,6.75,24.0,,"6ND proxy; 25B active × [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/CohereLabs/command-a-plus-05-2026-bf16""]",,,,0,6.75,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,command-aplus,Separate-generation assumption; no Vision/Translate/North increments added.,True,"[2.25, 3.75, 6.0]" | |
| xAI,Grok 2 — 2024 base scenario,C,0,9.763200000000001,40.68,,"6ND proxy; 113B active × [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/xai-org/grok-2""]",,,,0,9.763200000000001,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,grok2,Grok1 separately counted; active113B is config-derived approximation; independent-run assumption; no additional instruct run.,True,"[2.712, 5.424, 10.17]" | |
| SK Telecom,A.X K1,C,1.584,6.4152000000000005,23.76,,"6ND proxy; 33B active × [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/skt/A.X-K1""]",,,,0,6.4152000000000005,,No actual dollar invoice; price and FLOP conversion assumptions. ,axk1,Card explicitly says from scratch; A.X3.1 omitted.,False,"[1.98, 3.564, 5.94]" | |
| SK Telecom,A.X K2 — pretraining,B,1.29888,2.9224799999999997,6.4944,,"6ND proxy; 33B active × [8.2, 8.2, 8.2]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/skt/A.X-K2""]",,,,2.9224799999999997,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,axk2,Card explicitly says from scratch; K1 not reused as a charged base. Posttraining not counted.,False,"[1.6236, 1.6236, 1.6236]" | |
| Alibaba,Qwen 2 — dense 0.5/1.5/7/72B,B,2.7336,6.1506,13.668,,"Reported schedule: 0.5B×12T + (1.5+7+72)B×7T; 6ND; prices unchanged. 7T is a conservative reported slice, excludes long-context overhead.","[""https://arxiv.org/html/2407.10671v1""]",,,,6.1506,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,qwen2-dense,Pre-Qwen2.5 generation; each dense size once; excludes Qwen1/1.5 unresolved overlap.,False,"[3.417, 3.417, 3.417]" | |
| Alibaba,Qwen 2 57B-A14B — upcycling increment,B,0.3024,0.6804,1.512,,"6ND proxy; 14B active × [4.5, 4.5, 4.5]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","[""https://arxiv.org/html/2407.10671v1""]",,,,0.6804,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,qwen2-moe,Only additional4.5T after Qwen2-7B; parent is already in preceding dense entry.,False,"[0.378, 0.378, 0.378]" | |
| Z.ai,GLM 4 32B — April 2025 base,B,2.304,5.184,11.52,,"6ND proxy; 32B active × [15, 15, 15]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/zai-org/GLM-4-32B-Base-0414""]",,,,5.184,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,glm4-32,Distinct from GLM4.5 and5; Z1/Rumination/Flash/vision not added.,False,"[2.88, 2.88, 2.88]" | |
| ByteDance Seed,Seed-OSS 36B — base,B,2.0736000000000003,4.6656,10.368,,"6ND proxy; 36B active × [12, 12, 12]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/ByteDance-Seed/Seed-OSS-36B-Base""]",,,,4.6656,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,seedoss,"One base only; woSyn alternative, Seed-Coder and BAGEL excluded.",False,"[2.592, 2.592, 2.592]" | |
| Meta,Chameleon 7B + 34B — pretraining,B,5.138888,10.277776,15.416664,,"5138888 GPU-hours × [1, 2, 3] USD/GPU-hour (assumed prices).","[""https://arxiv.org/html/2405.09818v1""]",,,,10.277776,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,chameleon,Separate early-fusion runs; no Llama base charged.,False, | |
| Meta,Code Llama — released 12-model increment,B,1.4,2.8,4.2,,"1400000 GPU-hours × [1, 2, 3] USD/GPU-hour (assumed prices).","[""https://huggingface.co/meta-llama/CodeLlama-70b-hf""]",,,,2.8,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,codellama,"Llama2 upstream already in V3; all12 includes70B. Older400k report is subset, not extra.",False, | |
| Meta,CWM — pretraining + mid-training,B,1.9968000000000001,4.4928,9.984,,"6ND proxy; 32B active × [13, 13, 13]T tokens; tokens reported; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/facebook/cwm-pretrain""]",,,,4.4928,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,cwm,8T+5T once across pretrain/SFT/final checkpoints; RL/SFT extra excluded.,False,"[2.496, 2.496, 2.496]" | |
| AI21 Labs,Jamba 1.5 Large — base scenario,C,0,8.121599999999999,33.84,,"6ND proxy; 94B active × [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://arxiv.org/html/2408.12570v1""]",,,,0,8.121599999999999,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,jamba15,One Large lineage;1.6/1.7/Mini/Jamba2 not separately costed.,True,"[2.256, 4.512, 8.46]" | |
| StepFun,Step 3 — base scenario,C,0,7.3872,27.36,,"6ND proxy; 38B active × [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/stepfun-ai/step3""]",,,,0,7.3872,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,step3,"V3 Step5 explicitly excluded Step3; distinct-run assumption, vision extra excluded.",True,"[2.28, 4.104, 6.84]" | |
| StepFun,Step 3.5 Flash — base scenario,C,0,2.9699999999999998,10.56,,"6ND proxy; 11B active × [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/stepfun-ai/Step-3.5-Flash-Base""]",,,,0,2.9699999999999998,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,step35,"Separate base assumed, not proven from architecture.3.7 and midtrain checkpoint not charged separately.",True,"[0.99, 1.65, 2.64]" | |
| Ai2,OLMo 3 — 7B + 32B pretraining,B,1.0440479999999999,2.3491079999999998,5.2202399999999995,,Reported5.93T/5.50T; nominal7B/32B;6ND. Later stages omitted.,"[""https://huggingface.co/allenai/Olmo-3-1125-32B""]",,,,2.3491079999999998,0,,No actual dollar invoice; price and FLOP conversion assumptions. ,olmo3,OLMo2 allowance limited to OLMo2; OLMo3 treated as new pretraining generation.,False,"[1.3050599999999999, 1.3050599999999999, 1.3050599999999999]" | |
| Mistral AI,Mistral Small 4 — base scenario,C,0,1.755,6.24,,"6ND proxy; 6.5B active × [15, 25, 40]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/mistralai/Mistral-Small-4-119B-2603""]",,,,0,1.755,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,mistral-small4,Independent/upcycled status unknown. Leanstral not added.,True,"[0.585, 0.975, 1.56]" | |
| Mistral AI,Mistral Small 3 — 24B base scenario,C,0,4.6656,17.28,,"6ND proxy; 24B active × [10, 18, 30]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/mistralai/Mistral-Small-24B-Base-2501""]",,,,0,4.6656,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,mistral-small3,"One24B lineage;Small3.1/3.2,Magistral,DevstralSmall,VoxtralSmall not separately added.",True,"[1.44, 2.592, 4.32]" | |
| Mistral AI,Mistral NeMo — 12B joint base,C,0.2304,1.0368,4.32,,"6ND proxy; 12B active × [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/mistralai/Mistral-Nemo-Base-2407""]",,,,0,1.0368,,No actual dollar invoice; price and FLOP conversion assumptions. ,mistral-nemo,Joint NVIDIA/Mistral model counted once under Mistral; NVIDIA mirror and Pixtral not additional base.,False,"[0.288, 0.576, 1.08]" | |
| Mistral AI,Mistral 7B — original base,C,0.05184000000000001,0.1944,0.6912,,"6ND proxy; 7.2B active × [1.5, 2.5, 4]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/mistralai/Mistral-7B-v0.1""]",,,,0,0.1944,,No actual dollar invoice; price and FLOP conversion assumptions. ,mistral7,v0.2/v0.3/Mathstral not separately added; Mamba excluded.,False,"[0.06480000000000001, 0.108, 0.1728]" | |
| Mistral AI,Codestral 22B — one base scenario,C,0,1.9180799999999998,7.991999999999999,,"6ND proxy; 22.2B active × [4, 8, 15]T tokens; token volumes ASSUMED; $0.8/1.8/4M per 1e24 FLOP.","[""https://huggingface.co/mistralai/Codestral-22B-v0.1""]",,,,0,1.9180799999999998,,No actual dollar invoice; price and FLOP conversion assumptions. Independent-run scenario; reuse/upcycling not excluded.,codestral,Small2409 may share base: not added; Ministral8B2410 excluded.,True,"[0.5327999999999999, 1.0655999999999999, 1.9979999999999998]" | |