Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Paper
•
2304.01373
•
Published
•
9
Pythia Scaling Suite
Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia
Note The paper introduced the Pythia model suite
EleutherAI/pythia-14m
Text Generation • 39.2M • Updated • 79.2k • 25Note This model was trained after the paper was released.
EleutherAI/pythia-70m
95.6M • Updated • 73.7k • 74Note This model was trained after the paper was released.
-
EleutherAI/pythia-160m
Text Generation • 0.2B • Updated • 79.2k • 35 -
EleutherAI/pythia-410m
Text Generation • 0.5B • Updated • 39.3k • 35 -
EleutherAI/pythia-1b
Text Generation • 1B • Updated • 25.3k • 42 -
EleutherAI/pythia-1.4b
Text Generation • 2B • Updated • 16.7k • 25 -
EleutherAI/pythia-2.8b
Text Generation • 3B • Updated • 51.5k • 31 -
EleutherAI/pythia-6.9b
Text Generation • 7B • Updated • 29.9k • 57 -
EleutherAI/pythia-12b
Text Generation • 12B • Updated • 5.9k • 141 -
EleutherAI/pythia-70m-deduped
Text Generation • 95.6M • Updated • 230k • 25 -
EleutherAI/pythia-160m-deduped
Text Generation • 0.2B • Updated • 36.1k • 3 -
EleutherAI/pythia-410m-deduped
Text Generation • 0.5B • Updated • 13.7k • 20 -
EleutherAI/pythia-1b-deduped
Text Generation • 1B • Updated • 6.16k • 19 -
EleutherAI/pythia-1.4b-deduped
Text Generation • Updated • 11.3k • 21 -
EleutherAI/pythia-2.8b-deduped
Text Generation • 3B • Updated • 3.96k • 14 -
EleutherAI/pythia-6.9b-deduped
Text Generation • Updated • 19.4k • 8 -
EleutherAI/pythia-12b-deduped
Text Generation • Updated • 3.48k • 52