Issue 2026-11-11 · Voices
Dwarkesh Patel @dwarkesh_sp
This got totally buried cause it came out the same day as Navier Stokes. But I think this is super interesting evidence about what the true major source of algorithmic progress is. [引用 @dwarkesh_sp]: Pretraining progress seems to be coming mostly from data improvements. @who_is_jerbear and I pretrained combinations of year-representative open model recipes and data corpuses across 2019 to 2025 at various small scales. Data improvements contributed 3.24x as many compute multipliers as model improvements did (12.0x vs 3.7x). And the gains stack independently - a better dataset helps every ar…
Read original ↗Topics:Dwarkesh Patel