用平滑梯度上升提升大模型删知识时的稳定性与效果
Label Smoothing Improves Gradient Ascent in LLM Unlearning
- 将遗忘数据与构造的正常数据混合平滑，改进梯度上升方向
- 在3个基准上均优于原始方法，关键指标达前两名
- 适合需要安全删减敏感知识的模型优化场景
大模型删知识已成为一种有前景的方法，旨在以低成本使模型遗忘有害或不希望保留的知识，同时尽可能保持模型可用性。现有方法中，最直接的是对遗忘数据执行梯度上升（GA），迫使模型遗忘该数据集。然而，GA存在严重不稳定性，因其更新方向发散，常导致模型性能急剧下降。为此，我们提出平滑梯度上升（SGA）：通过可调平滑率将遗忘数据与多个构造的正常数据结合。直观上，这将GA从仅在遗忘数据上学习扩展为在遗忘与正常数据上联合学习，实现更稳定的删知识过程，并更好保留模型性能。理论上，我们提供了最优平滑率选择的指导。实验在TOFU、Harry Potter和MUSE-NEWS三个基准上验证，结果表明SGA在所有指标上持续优于原始GA方法，在多个关键指标上达到前两名。
原文摘要 · Abstract (English)
LLM unlearning has emerged as a promising approach, aiming to enable models to forget hazardous/undesired knowledge at low cost while preserving as much model utility as possible. Among existing techniques, the most straightforward method is performing Gradient Ascent (GA) w.r.t. the forget data, thereby forcing the model to unlearn the forget dataset. However, GA suffers from severe instability, as it drives updates in a divergent direction, often resulting in drastically degraded model utility. To address this issue, we propose Smoothed Gradient Ascent (SGA). SGA combines the forget data with multiple constructed normal data through a tunable smoothing rate. Intuitively, this extends GA from learning solely on the forget data to jointly learning across both forget and normal data, enabling more stable unlearning while better preserving model utility. Theoretically, we provide the theoretical guidance on the selection of the optimal smoothing rate. Empirically, we evaluate SGA on three benchmarks: TOFU, Harry Potter, and MUSE-NEWS. Experimental results demonstrate that SGA consistently outperforms the original Gradient Ascent (GA) method across all metrics and achieves top-2 performance among all baseline methods on several key metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。