Percy Liang@percyliang
Synthetic data is used all the time now for LM pretraining...but can we carefully quantify the gains in compute efficiency and data efficiency? [Quoting @AhmedSQRD]: Synthetic rewrites are now standard to improve data quality and when data is limited. But how…