提出在线停止的高效遗忘方法,省去冗余训练和评估。
Fast Unlearning at Scale via Margin Self-Correction
- 通过缩小目标词与替代词的逻辑差距实现主动遗忘
- 在多个数据集上以极低计算成本达成良好遗忘-保留平衡
- 适合大规模模型快速遗忘敏感数据,无需额外验证
语言模型的遗忘更新旨在让训练好的模型表现得像从未见过某些训练样本一样,同时保持性能并避免昂贵的重新训练。现有方法通常在固定预算下微调预训练模型,并通过下游验证数据评估多个保存的检查点来选择最终模型。这一过程存在两个计算浪费:训练超出理想的遗忘-保留权衡,以及需要额外存储和重复评估的检查点选择。为此,我们提出边际自校正(MASC),一种具有在线停止规则的高效遗忘方法,无需下游评估。给定需遗忘的文本序列,MASC主动减小原始下一个词与其最可能替代项之间的逻辑差距。当该差距在所有遗忘序列中足够多的位置平均变小时,即输出最终模型。在TOFU、MUSE News和MUSE Books数据集上,MASC以远低于现有基线的计算开销实现了有竞争力的遗忘-保留权衡。进一步观察发现,随着模型规模增大(参数量增加),MASC和SimNPO的权衡均改善:遗忘指标保持相当,而保留性能提升。
原文摘要 · Abstract (English)
Language-model unlearning updates a trained model to behave as if it had not seen selected training examples, while preserving utility and avoiding costly retraining. Existing approaches typically fine-tune the pretrained model with a fixed training budget and select the final model afterwards by evaluating several saved checkpoints on downstream validation data. Two sources of unnecessary computation limit scalability: training beyond the desired forget-retain trade-off, and checkpoint selection that requires extra storage and repeated evaluations. To address these limitations, we introduce MArgin Self-Correction (MASC), an efficient unlearning method with an online stopping rule that does not require downstream evaluation. Given a text sequence to be forgotten, MASC actively reduces the logit gap between the original next token and the most likely alternatives. It outputs a final model once this gap is small on average over a sufficiently large proportion of token positions across all forget sequences. On TOFU, MUSE News, and MUSE Books, MASC achieves a competitive forget-retain trade-off at a fraction of the computational cost of existing baselines. We further observe that as we increase model size (a.k.a. number of parameters), the trade-offs improve for both MASC and SimNPO -- the forget metrics remain comparable while retain utility increases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。