Pythia 160M, Mamba 130M, and RWKV 169M models trained on OpenWebText for 4000 steps (context window: 1024; effective batch size: 512). 6 seeds each.
James Michaelov
jmichaelov
AI & ML interests
None yet
Recent Activity
published a dataset 2 days ago
jmichaelov/koi_fixed updated a dataset 2 days ago
jmichaelov/koi_fixed updated a model 17 days ago
jmichaelov/fasttext-en-wikiOrganizations
models 54
jmichaelov/fasttext-en-wiki
Updated
jmichaelov/pythia31m_b128_constant_seed0
30.5M • Updated • 1
jmichaelov/pythia31m_b16_constant_seed0
30.5M • Updated • 1
jmichaelov/pythia31m_b32_constant_seed2
30.5M • Updated • 1
jmichaelov/pythia31m_b1024_64_constant_seed1
30.5M • Updated • 1
jmichaelov/pythia31m_b512_constant_seed1
30.5M • Updated • 1
jmichaelov/pythia31m_b256_constant_seed2
30.5M • Updated • 1
jmichaelov/pythia31m_b64_constant_seed1
30.5M • Updated • 1
jmichaelov/pythia31m_b16_constant_seed3
30.5M • Updated • 1
jmichaelov/pythia31m_b128_constant_seed3
30.5M • Updated • 1
datasets 14
jmichaelov/koi_fixed
Viewer • Updated • 1.25k • 6
jmichaelov/bhs
Viewer • Updated • 22k • 69 • 1
jmichaelov/blimp_nl
Viewer • Updated • 8.4k • 136
jmichaelov/lm_syneval
Viewer • Updated • 158k • 46
jmichaelov/inverse_scaling_prize-hindsight_neglect
Viewer • Updated • 315 • 4
jmichaelov/inverse_scaling_prize-memo_trap
Viewer • Updated • 936 • 15
jmichaelov/inverse_scaling_prize-neqa
Viewer • Updated • 300 • 4
jmichaelov/inverse_scaling_prize-redefine
Viewer • Updated • 1.24k • 4
jmichaelov/inverse_scaling_prize-into_the_unknown
Viewer • Updated • 1.82k • 4
jmichaelov/inverse_scaling_prize-modus_tollens
Viewer • Updated • 1.24k • 17