arXiv:2607.23970cs.LGcs.AI2026-07被引 5

用模式连通性解析机器遗忘,揭示遗忘路径与隐私差异

Understanding Machine Unlearning Through the Lens of Mode Connectivity

  • 通过模式连通性分析遗忘路径,发现遗忘模型常处于平滑低损失连接区
  • 不同遗忘方法的模型在参数空间中存在显著隐私差异,且遗忘过程非线性
  • 适用于研究模型可解释性、隐私保护及对抗重训练攻击的工程师

机器遗忘旨在不从头训练的情况下移除模型中的不当信息。尽管近期取得进展,但遗忘过程的损失景观与优化几何仍不清楚。本文通过模式连通性视角研究机器遗忘——即独立训练的模型通常能在参数空间中通过平滑低损失路径相连。我们提出「遗忘中的模式连通性」(MCU),在课程学习、二阶优化及不同遗忘方法间进行评估。结果表明,许多遗忘模型位于具有平滑保留/遗忘行为的连通基域内;训练动态变化会将解移至不同基域。MCU还揭示:同一基域内的模型在隐私指标上可有显著差异,且遗忘进程从原始模型到遗忘模型呈非线性。此外,线性连通性表明多数近似遗忘方法与重训练机制本质不同。基于MCU的集成可提升泛化能力与抗重训练攻击鲁棒性,且MCU平滑度与遗忘难度相关。这是首个从模式连通性视角研究机器遗忘的工作。

原文摘要 · Abstract (English)

Machine Unlearning aims to remove undesired information from trained models without full retraining from scratch. Despite recent progress, the loss landscape and optimization geometry of unlearning are poorly understood. In this paper, we study machine unlearning through the lens of mode connectivity--the phenomenon that independently trained models can often be connected by smooth low-loss paths in parameter space. We introduce {\em mode connectivity in unlearning} (MCU) and evaluate it across a range of settings, including curriculum learning, second-order optimization, and connectivity across different unlearning methods. We find that many unlearned models lie in connected basins with smooth retain/forget behavior, while changes in training dynamics can move solutions into different basins. MCU also reveals that models within the same basin can differ substantially on privacy metrics, and that unlearning progresses nonlinearly from the original model to the unlearned model. In addition, linear connectivity suggests that most approximate unlearning methods are mechanistically distinct from retraining. Finally, MCU-based ensembling can improve generalization and robustness to relearning attacks, and MCU smoothness correlates with unlearning difficulty. To our knowledge, this is the first study of machine unlearning through the lens of mode connectivity.

机器遗忘模式连通性隐私保护优化几何

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。