TL;DR of the paper, generated automatically.

Zeroing under 0.05% of an LLM's weights matches full fine-tuning at scale.

• Method: Deletion Tuning masks gate projections, with no other weights changed.
• Size: the mask is hundreds to thousands of times smaller than FFT checkpoints.
• Forgetting: under RL, FFT and LoRA lose up to 5.4 prior-task points.