The A.I. Daily
Soft-MTP method improves multi-token prediction training
Researchers show the standard multi-token prediction (MTP) gradient hides a useful KL self-distillation signal behind a noisy residual that hurts next-token prediction. Their Soft-MTP method needs only a one-line code change to separate that signal and improves MTP across several tasks; code and paper are released.