提出首个无需重训的机器删忆算法，可高效清除模型中的数据痕迹。
Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions
- 通过回溯训练过程中的早期步骤，实现对非凸损失函数的删忆。
- 在不重新训练的前提下，证明了(ε, δ)级隐私保障与性能权衡。
- 适用于预训练模型，适合需要合规删忆的工业应用。
机器删忆旨在不从头训练的情况下高效移除模型中的数据，以处理污染或过时数据，或满足用户‘被遗忘的权利’。认证机器删忆基于差分隐私提供强理论保障，量化算法从模型权重中擦除数据的程度。现有研究多局限于凸或强凸损失函数，或对非凸目标施加严苛假设。本文首次提出一种一阶、黑盒（可应用于用普通梯度下降预训练的模型）的删忆算法，适用于一般非凸损失函数。该方法通过‘回溯’训练过程中的早期步骤，再对保留数据点进行梯度下降，实现删忆。我们证明了(ε, δ)级别的认证删忆与性能保证，建立了隐私-效用-复杂度之间的权衡关系，并对满足Polyak-Lojasiewicz不等式的函数给出了泛化保证。最后，我们在新设计的实验框架下验证了该算法优于现有方法，在更贴近实际删忆场景中表现更优。
原文摘要 · Abstract (English)
Machine unlearning algorithms aim to efficiently remove data from a model without retraining it from scratch, in order to remove corrupted or outdated data or respect a user's ``right to be forgotten." Certified machine unlearning is a strong theoretical guarantee based on differential privacy that quantifies the extent to which an algorithm erases data from the model weights. In contrast to existing works in certified unlearning for convex or strongly convex loss functions, or nonconvex objectives with limiting assumptions, we propose the first, first-order, black-box (i.e., can be applied to models pretrained with vanilla gradient descent) algorithm for unlearning on general nonconvex loss functions, which unlearns by ``rewinding" to an earlier step during the learning process before performing gradient descent on the loss function of the retained data points. We prove $(ε, δ)$ certified unlearning and performance guarantees that establish the privacy-utility-complexity tradeoff of our algorithm, and we prove generalization guarantees for functions that satisfy the Polyak-Lojasiewicz inequality. Finally, we demonstrate the superior performance of our algorithm compared to existing methods, within a new experimental framework that more accurately reflects unlearning user data in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。