Percy Liang @percyliang

Synthetic data is used all the time now for LM pretraining...but can we carefully quantify the gains in compute efficiency and data efficiency? [引用 @AhmedSQRD]: Synthetic rewrites are now standard to improve data quality and when data is limited. But how much compute should you spend rewriting? And can a model trained on rewrites beat the rewriting model? We fit scaling laws to answer both: 1/🧵 https://t.co/IuYCwIP8G1