Issue 2026-11-10 · Industry · 研究 · 开源
Study: deleting under 0.05% of weights matches full fine-tuning
Researchers propose Deletion Tuning, which zeroes fewer than 0.05% of weights in transformer gate projections and matches full fine-tuning across reinforcement learning, supervised fine-tuning and on-policy distillation. The learned masks are hundreds to thousands of times smaller than full checkpoints and forget far less in continual learning.
Read original ↗Topics:Deletion Tuning