arXiv:2602.01703cs.LGcs.CL2026-02ACL被引 1

提出AGT^AO框架,实现大模型安全删忆且不丢性能。

$\textbf{AGT$^{AO}$}$: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality

  • 用自适应正交机制缓解遗忘与保留的梯度冲突。
  • 通过对抗门控训练使删忆效果达KUR≈0.01,保持MMLU 58.30得分。
  • 适合需隐私保护的高可靠性场景,如医疗、金融领域应用。

大型语言模型虽能力卓越,却可能无意中记忆敏感数据,带来严重隐私与安全风险。机器删忆对缓解此类风险至关重要,但现有方法面临根本矛盾:激进删忆常导致灾难性遗忘,损害模型可用性;保守策略又易造成表面遗忘,模型仍可能被对抗恢复。为此,我们提出AGT^AO(对抗门控训练结合自适应正交),一种统一框架,旨在平衡彻底删忆与性能保留。该方法引入自适应正交(AO)动态缓解遗忘与保留目标间的几何梯度冲突,从而最小化意外知识损失;同时,对抗门控训练(AGT)将删忆建模为潜在空间的极小极大博弈,采用课程式门控机制模拟并抵御内部恢复尝试。大量实验表明,AGT^AO在删忆效果(KUR≈0.01)与模型性能(MMLU 58.30)间取得优异权衡。代码已公开于https://github.com/TiezMind/AGT-unlearning。

原文摘要 · Abstract (English)

While Large Language Models (LLMs) have achieved remarkable capabilities, they unintentionally memorize sensitive data, posing critical privacy and security risks. Machine unlearning is pivotal for mitigating these risks, yet existing paradigms face a fundamental dilemma: aggressive unlearning often induces catastrophic forgetting that degrades model utility, whereas conservative strategies risk superficial forgetting, leaving models vulnerable to adversarial recovery. To address this trade-off, we propose $\textbf{AGT$^{AO}$}$ (Adversarial Gating Training with Adaptive Orthogonality), a unified framework designed to reconcile robust erasure with utility preservation. Specifically, our approach introduces $\textbf{Adaptive Orthogonality (AO)}$ to dynamically mitigate geometric gradient conflicts between forgetting and retention objectives, thereby minimizing unintended knowledge degradation. Concurrently, $\textbf{Adversarial Gating Training (AGT)}$ formulates unlearning as a latent-space min-max game, employing a curriculum-based gating mechanism to simulate and counter internal recovery attempts. Extensive experiments demonstrate that $\textbf{AGT$^{AO}$}$ achieves a superior trade-off between unlearning efficacy (KUR $\approx$ 0.01) and model utility (MMLU 58.30). Code is available at https://github.com/TiezMind/AGT-unlearning.

大模型删忆隐私保护对抗训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。