Study: PBT doesn't speed grokking but escapes dead zones

An experiment on modular addition found Population Based Training over learning rate and weight decay was slower than grid search: the best fixed cell hit 95% validation accuracy in about 317 steps versus about 2,667 for PBT. From a dead-zone start where no fixed setting grokked, PBT still succeeded 3/3 in roughly 6,850 steps. The author calls PBT a rescue tool, not a speed-up.