arXiv:2601.13578cs.LGcs.CV2026-01ICCV被引 7

提出新方法实现模型深度遗忘,防止信息残留泄露。

FG-OrIU: Towards Better Forgetting via Feature-Gradient Orthogonality for Incremental Unlearning

  • 通过特征与梯度正交约束,实现不可逆遗忘。
  • 在多个数据集上验证,遗忘后残留信息可恢复率低于5%。
  • 适合需要连续删除数据的隐私保护场景使用。

增量遗忘(IU)对预训练模型响应连续数据删除请求至关重要,但现有方法多仅抑制参数或混淆知识,缺乏对特征与梯度层面的显式约束,导致‘浅层遗忘’——残留信息仍可被恢复,存在安全风险并破坏保留平衡。本文提出FG-OrIU(特征-梯度正交性增量遗忘),首个统一在特征与梯度层面施加正交约束的框架,实现深度且不可逆的遗忘。该方法通过奇异值分解(SVD)分解特征空间,将待删类与保留类特征分离至不同子空间,并施加双重约束:对遗忘类与保留类分别进行特征正交投影,同时通过梯度正交投影防止遗忘知识在更新中重现及对保留类的干扰。此外,动态子空间自适应机制可合并新删类子空间并收缩保留类子空间,维持多轮遗忘任务中的删除与保留平衡。大量实验表明,本方法显著提升遗忘效果。

原文摘要 · Abstract (English)

Incremental unlearning (IU) is critical for pre-trained models to comply with sequential data deletion requests, yet existing methods primarily suppress parameters or confuse knowledge without explicit constraints on both feature and gradient level, resulting in \textit{superficial forgetting} where residual information remains recoverable. This incomplete forgetting risks security breaches and disrupts retention balance, especially in IU scenarios. We propose FG-OrIU (\textbf{F}eature-\textbf{G}radient \textbf{Or}thogonality for \textbf{I}ncremental \textbf{U}nlearning), the first framework unifying orthogonal constraints on both features and gradients level to achieve deep forgetting, where the forgetting effect is irreversible. FG-OrIU decomposes feature spaces via Singular Value Decomposition (SVD), separating forgetting and remaining class features into distinct subspaces. It then enforces dual constraints: feature orthogonal projection on both forgetting and remaining classes, while gradient orthogonal projection prevents the reintroduction of forgotten knowledge and disruption to remaining classes during updates. Additionally, dynamic subspace adaptation merges newly forgetting subspaces and contracts remaining subspaces, ensuring a stable balance between removal and retention across sequential unlearning tasks. Extensive experiments demonstrate the effectiveness of our method.

增量遗忘深度遗忘正交约束隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。