arXiv:2507.09786cs.LG2025-07

提出两种方法加速模型遗忘,显著降低计算时间。

Leveraging Distribution Matching to Make Approximate Machine Unlearning Faster

  • 用分布匹配合并相似图像,大幅压缩需保留数据量
  • 通过强化损失函数和分布对齐,加快遗忘收敛速度
  • 适合需要高效隐私保护的机器学习场景

近似机器遗忘(AMU)可通过在保留的数据子集上特殊微调,使模型‘遗忘’特定训练数据。然而,处理大规模保留数据仍占主导计算时间,且减少遗忘轮次仍是挑战。本文提出两种互补方法以加速任意面向分类任务的AMU方法:首先,提出一种新型分布匹配数据压缩(Blend),通过合并视觉相似图像并共享混合权重,显著缩小保留数据集规模,预处理开销小,速度比现有最优方法快数个数量级;其次,提出以损失为中心的加速方法(A-AMU),通过增强主损失项加速遗忘,并引入可微正则化项,匹配被遗忘数据与未见分布数据的损失分布。大量实验表明,该数据与损失双优化策略在单轮与多轮场景下均显著降低端到端遗忘延迟,同时保持模型性能与隐私性。据我们所知,这是首个系统性地结合专用数据压缩技术与专用加速损失函数来提升遗忘效率的工作。代码已开源。

原文摘要 · Abstract (English)

Approximate machine unlearning (AMU) enables models to `forget' specific training data through specialized fine-tuning on a retained (and forget) subset of training set. However, processing this large retained subset still dominates computational runtime, while reductions of unlearning epochs also remain a challenge. In this paper, we propose two complementary methods to accelerate arbitrary classification-oriented AMU method. First, \textbf{Blend}, a novel distribution-matching dataset condensation (DC), merges visually similar images with shared blend-weights to significantly reduce the retained set size. It operates with minimal pre-processing overhead and is orders of magnitude faster than state-of-the-art DC methods. Second, our loss-centric method, \textbf{Accelerated-AMU (A-AMU)}, augments the AMU objective to quicken convergence. A-AMU achieves this by combining a steepened primary loss to expedite forgetting with a differentiable regularizer that matches the loss distributions of forgotten and in-distribution unseen data. Our extensive experiments demonstrate that this dual approach of data and loss-centric optimization dramatically reduces end-to-end unlearning latency across both single and multi-round scenarios, all while preserving model utility and privacy. To our knowledge, this is the first work to systematically tackle unlearning efficiency by jointly designing a specialized dataset condensation technique with a dedicated accelerated loss function. Code is available at https://github.com/algebraicdianuj/DC_Unlearning.

机器遗忘数据压缩损失优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。