Percy Liang @percyliang

Given how many people have tried their hand at NanoGPT, I really didn't expect a 40% reduction in a single go! [引用 @classiclarryd]: New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with incredibly clever engineering and ML judgement. If a flop is low value on a particular step, skip it. Specifically: -(~8s…