Deletion Tuning

1 stories

Every Deletion Tuning story collected by The AI Daily, 1 so far, newest first, refreshed hourly.

Study: deleting under 0.05% of weights matches full fine-tuning

Researchers propose Deletion Tuning, which zeroes fewer than 0.05% of weights in transformer gate projections and matches full fine-tuning across reinforcement learning, supervised fine-tuning and on-policy distillation. The learned masks are hundreds to thousands of times smaller than full checkpoints and forget far less in continual learning.

r/LocalLLaMA ·
That is everything