提出新方法让大模型真正遗忘敏感信息，防止知识复现
Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning
- 通过归因引导正则化，精准移除目标知识而非伪装隐藏
- 在两种重训练场景下，遗忘率超基线15%以上，复现率极低
- 适合需高安全性的模型部署，如医疗、金融领域应用
大规模语言模型在海量网络数据上训练时可能记住隐私或敏感信息，带来严重隐私风险。尽管已有遗忘方法缓解此类问题，但在后续训练中仍存在“再学习”现象，导致大量被遗忘知识重新浮现。本文揭示，现有遗忘方法仅实现浅层对齐：并非真正擦除目标知识，而是生成虚假的遗忘神经元，通过放大负面干扰来掩盖知识。为此，我们提出Ssiuu方法，采用归因引导正则化机制，防止虚假负向影响，确保目标知识被真实删除。实验表明，该方法在两种实际重训练场景中均表现优异：(1) 敏感数据对抗性注入，(2) 使用指令遵循基准的良性攻击。结果验证了其在可靠遗忘与抗复现方面的显著优势，强调了鲁棒且忠实遗忘方法在语言模型安全部署中的必要性。
原文摘要 · Abstract (English)
Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks. Although some unlearning methods mitigate these risks, they remain vulnerable to "relearning" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface. In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence to hide it. To overcome this limitation, we introduce Ssiuu, a new class of unlearning methods that employs attribution-guided regularization to prevent spurious negative influence and faithfully remove target knowledge. Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an instruction-following benchmark. Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。