揭示机器遗忘的计算复杂性边界,明确何时可高效删数据。
When to Forget? Complexity Trade-offs in Machine Unlearning
- 构建遗忘复杂度比新指标,量化遗忘与重训练的计算代价
- 发现三种不同阶段:不可行、噪声即够、显著节省计算
- 指出维度、遗忘样本数和隐私约束决定遗忘是否实用
机器遗忘旨在移除特定数据点对已训练模型的影响,目标是以远低于全量重训练的成本实现。本文分析遗忘方法的效率,首次建立该问题的最小最大计算时间上下界,刻画最优算法在最难目标函数下的性能表现。针对强凸目标函数,且假设遗忘数据不可访问的情形,我们提出遗忘复杂度比这一新指标,用于比较最优遗忘方法与全量重训练的计算开销。相图揭示三类不同区域:一类中以更低代价遗忘不可行;一类中添加噪声即可满足要求,遗忘极为简单;另一类中遗忘能显著优于重训练。这些发现凸显数据维度、需遗忘样本数量及隐私约束在决定遗忘实际可行性中的关键作用。
原文摘要 · Abstract (English)
Machine Unlearning (MU) aims at removing the influence of specific data points from a trained model, striving to achieve this at a fraction of the cost of full model retraining. In this paper, we analyze the efficiency of unlearning methods and establish the first upper and lower bounds on minimax computation times for this problem, characterizing the performance of the most efficient algorithm against the most difficult objective function. Specifically, for strongly convex objective functions and under the assumption that the forget data is inaccessible to the unlearning method, we provide a phase diagram for the unlearning complexity ratio -- a novel metric that compares the computational cost of the best unlearning method to full model retraining. The phase diagram reveals three distinct regimes: one where unlearning at a reduced cost is infeasible, another where unlearning is trivial because adding noise suffices, and a third where unlearning achieves significant computational advantages over retraining. These findings highlight the critical role of factors such as data dimensionality, the number of samples to forget, and privacy constraints in determining the practical feasibility of unlearning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。