Issue 2026-11-10 · Industry · 研究 · 开源

Study: deleting under 0.05% of weights matches full fine-tuning

Researchers propose Deletion Tuning, which zeroes fewer than 0.05% of weights in transformer gate projections and matches full fine-tuning across reinforcement learning, supervised fine-tuning and on-policy distillation. The learned masks are hundreds to thousands of times smaller than full checkpoints and forget far less in continual learning.

r/LocalLLaMA11 d ago
Read original ↗