Study: FLOP-based estimates poorly predict fine-tuning time
Using 272 production SFT jobs (~7,382 GPU-hours on NVIDIA H200), Hugging Face found the standard FLOP-based estimate explains little of observed runtime (R²=0.31, MAE 12.41h). A CatBoost regressor using only pre-dispatch information reached R²=0.98, MAE 1.71h and median error 0.51h in 5-fold CV. The result matters for scheduling shared GPU clusters.