thomwolf HF Staff commited on
Commit
fd9210b
·
verified ·
1 Parent(s): d7337b4

Plainer copy; state what is not counted

Browse files
Files changed (2) hide show
  1. README.md +3 -3
  2. index.html +23 -22
README.md CHANGED
@@ -9,8 +9,8 @@ pinned: false
9
  short_description: How much training compute is shared for free on the Hub?
10
  ---
11
 
12
- Hub Dollars estimates what it would cost to rent the GPUs that trained the open-weight models freely downloadable on the Hugging Face Hub: about $1B (central estimate; realistic range $0.82B to $1.35B), across 75 model families from 35 organizations, with a source for every number.
13
 
14
- Final training runs only, at GPU rental prices. Not included: experiments, research salaries, datasets and community fine-tunes.
15
 
16
- Data: `v4/families-v4.csv`, `v4/organizations-v4.csv`, `timeline/cumulative.csv`, `timeline/release-windows.csv`. The previous detailed audit (in French) is at `audit.html`.
 
9
  short_description: How much training compute is shared for free on the Hub?
10
  ---
11
 
12
+ Rough estimate of what it would cost to rent the GPUs that trained the open-weight models on the Hugging Face Hub: about $1B (10th to 90th percentile: $0.82B to $1.35B), for the final training runs of 75 model families from 35 organizations.
13
 
14
+ Not counted: community fine-tunes, LoRAs and merges, smaller families outside the list, experiments, salaries and datasets.
15
 
16
+ Data: `v4/families-v4.csv`, `v4/organizations-v4.csv`, `timeline/cumulative.csv`, `timeline/cumulative-realistic.csv`, `timeline/release-windows.csv`. Previous detailed audit (French): `audit.html`.
index.html CHANGED
@@ -63,35 +63,35 @@
63
  <main>
64
  <div class="kicker">Hub Dollars</div>
65
  <h1>About $1B of training compute is shared for free on the Hugging Face Hub</h1>
66
- <p class="lede">What it would cost to rent the GPUs that trained the open-weight models anyone can download today, counted model family by model family, with a source for every number.</p>
67
 
68
  <div class="tiles">
69
  <div class="tile"><b id="t-central">…</b><span>central estimate</span></div>
70
- <div class="tile"><b id="t-range">…</b><span>realistic range (10th to 90th percentile)</span></div>
71
- <div class="tile"><b id="t-fam">…</b><span>model families from <span id="t-org">…</span> organizations</span></div>
72
- <div class="tile"><b id="t-doc">…</b><span>backed by compute the developers reported themselves</span></div>
73
  </div>
74
 
75
- <h2>It keeps accelerating</h2>
76
- <p class="sub">Cumulative training cost of open-weight releases. The shaded band is the realistic range. Each family's cost is booked in one month: at its first or at its latest release.</p>
77
  <div class="card">
78
- <div class="controls">Book cost at
79
  <span class="seg" id="booking"><button data-v="late" class="on">latest release</button><button data-v="early">first release</button></span>
80
  </div>
81
  <svg id="curve" viewBox="0 0 1000 440" role="img" aria-label="Cumulative training compute cost over time"></svg>
82
  </div>
83
 
84
- <h2>Who paid for it</h2>
85
- <p class="sub">Central estimate per organization. The thin line shows the realistic range.</p>
86
  <div class="card"><svg id="orgs" role="img" aria-label="Estimated cost by organization"></svg></div>
87
 
88
- <h2>How solid are the numbers?</h2>
89
- <p class="sub">Share of the central estimate by the kind of evidence behind it.</p>
90
  <div class="evbar" id="evbar"></div>
91
  <div class="legend" id="evlegend"></div>
92
 
93
- <h2>Every model family</h2>
94
- <p class="sub">Click a column to sort. Costs in millions of USD, rental-equivalent.</p>
95
  <div class="tablebar">
96
  <input type="search" id="q" placeholder="Search a model or organization">
97
  <select id="ev"><option value="">All evidence</option><option value="A">Reported GPU-hours</option><option value="B">Reported size and tokens</option><option value="C">Partly assumed</option></select>
@@ -104,14 +104,15 @@
104
  </tr></thead><tbody></tbody>
105
  </table></div>
106
 
107
- <h2>How it is counted</h2>
108
  <ul class="method">
109
- <li><b>Final training runs only</b>, pre-training plus post-training where known, valued at GPU rental prices: centrally $2.5 per H100-hour, $2 per A100 or H800-hour, $5 per B200-hour. Where only FLOPs are known: $1.8M per 10²⁴ FLOP, which is H100 time at $2.5 per hour and 40% utilization.</li>
110
- <li><b>Compute comes from the developers when they publish it</b> (GPU-hours on model cards). Otherwise from 6 × active parameters × training tokens. When tokens are not published, they are assumed and the family is marked "partly assumed".</li>
111
- <li><b>No double counting.</b> A base model is counted once; official derivatives add only their extra training. Quantizations, mirrors and community fine-tunes are excluded.</li>
112
- <li><b>Not included:</b> failed runs and experiments, research salaries, datasets, and the long tail of millions of community uploads. A full R&amp;D cost would be several times higher.</li>
113
- <li><b>Realistic range: <span id="m-range"></span>.</b> A simulation that varies the GPU price ($2 to $3 per H100-hour, shared by all models), hardware utilization (35% to 50%) and each family's training quantities within its low and high estimates. Families with assumed token counts move together, since their guesses share the same prior. The range is the 10th to 90th percentile of 20,000 draws. Putting every assumption at its extreme at the same time would give <span id="m-extreme"></span>, which is not a realistic outcome.</li>
114
- <li>Release dates partly rely on proxies (Epoch AI publication dates).</li>
 
115
  </ul>
116
 
117
  <h2>Data</h2>
@@ -125,10 +126,10 @@
125
  <a href="v4/summary-realistic.json">Realistic range summary (JSON)</a>
126
  <a href="scripts/realistic_bounds.py">Simulation script</a>
127
  <a href="v4/summary.json">Summary (JSON)</a>
128
- <a href="audit.html">Full audit trail (previous version, in French)</a>
129
  </p>
130
 
131
- <footer>Estimate as of <span id="asof">October 2026</span>. Sources: model cards, technical reports and the <a href="https://epoch.ai/data">Epoch AI</a> model database. Corrections welcome in the Space discussions.</footer>
132
  </main>
133
  <div class="tip" id="tip"></div>
134
 
 
63
  <main>
64
  <div class="kicker">Hub Dollars</div>
65
  <h1>About $1B of training compute is shared for free on the Hugging Face Hub</h1>
66
+ <p class="lede">A rough estimate of what it would cost to rent the GPUs that trained the open-weight models on the Hub. It covers the final training runs of the main model families, not the long tail.</p>
67
 
68
  <div class="tiles">
69
  <div class="tile"><b id="t-central">…</b><span>central estimate</span></div>
70
+ <div class="tile"><b id="t-range">…</b><span>10th to 90th percentile</span></div>
71
+ <div class="tile"><b id="t-fam">…</b><span>model families, <span id="t-org">…</span> organizations</span></div>
72
+ <div class="tile"><b id="t-doc">…</b><span>from compute figures the developers published</span></div>
73
  </div>
74
 
75
+ <h2>Cumulative cost over time</h2>
76
+ <p class="sub">Each family's cost is placed at its first or latest release month. Shaded: 10th to 90th percentile.</p>
77
  <div class="card">
78
+ <div class="controls">Place cost at
79
  <span class="seg" id="booking"><button data-v="late" class="on">latest release</button><button data-v="early">first release</button></span>
80
  </div>
81
  <svg id="curve" viewBox="0 0 1000 440" role="img" aria-label="Cumulative training compute cost over time"></svg>
82
  </div>
83
 
84
+ <h2>By organization</h2>
85
+ <p class="sub">Bar: central estimate. Line: 10th to 90th percentile.</p>
86
  <div class="card"><svg id="orgs" role="img" aria-label="Estimated cost by organization"></svg></div>
87
 
88
+ <h2>Evidence behind the estimate</h2>
89
+ <p class="sub">Share of the central estimate by type of evidence.</p>
90
  <div class="evbar" id="evbar"></div>
91
  <div class="legend" id="evlegend"></div>
92
 
93
+ <h2>All model families</h2>
94
+ <p class="sub">Costs in $M. Click a column to sort.</p>
95
  <div class="tablebar">
96
  <input type="search" id="q" placeholder="Search a model or organization">
97
  <select id="ev"><option value="">All evidence</option><option value="A">Reported GPU-hours</option><option value="B">Reported size and tokens</option><option value="C">Partly assumed</option></select>
 
104
  </tr></thead><tbody></tbody>
105
  </table></div>
106
 
107
+ <h2>Method</h2>
108
  <ul class="method">
109
+ <li><b>Counted:</b> the final pre-training and post-training runs of 75 official model families.</li>
110
+ <li><b>Not counted:</b> community fine-tunes, LoRAs and merges (about 900k Hub repos declare a parent model), smaller families outside the list, experiments and failed runs, salaries, datasets. So this is a floor on what was spent, not the full cost.</li>
111
+ <li><b>Prices:</b> $2.5 per H100-hour, $2 per A100 or H800-hour, $5 per B200-hour. When only FLOPs are known, $1.8M per 10²⁴ FLOP (H100 at $2.5/hour, 40% utilization).</li>
112
+ <li><b>Compute:</b> GPU-hours from the model card when published. Otherwise 6 × active parameters × training tokens. When tokens aren't published they are guessed, and the family is marked "partly assumed".</li>
113
+ <li><b>Overlap:</b> a base model is counted once. Official variants only add their extra training. Quantizations and mirrors count as zero.</li>
114
+ <li><b>Range (<span id="m-range"></span>):</b> 20,000 simulated runs varying the GPU price ($2 to $3 per H100-hour), utilization (35% to 50%) and each family's training quantities between its low and high estimates. Families with guessed token counts move together. Setting every assumption to its extreme at once gives <span id="m-extreme"></span>.</li>
115
+ <li><b>Dates:</b> some release months come from Epoch AI publication dates.</li>
116
  </ul>
117
 
118
  <h2>Data</h2>
 
126
  <a href="v4/summary-realistic.json">Realistic range summary (JSON)</a>
127
  <a href="scripts/realistic_bounds.py">Simulation script</a>
128
  <a href="v4/summary.json">Summary (JSON)</a>
129
+ <a href="audit.html">Previous detailed audit (French)</a>
130
  </p>
131
 
132
+ <footer>Estimate as of <span id="asof">October 2026</span>. Sources: model cards, technical reports and the <a href="https://epoch.ai/data">Epoch AI</a> model database. If a number is wrong, open a discussion on the Space.</footer>
133
  </main>
134
  <div class="tip" id="tip"></div>
135