RAUL通过多目标优化实现精准删样，避免模型遗忘关键知识。
Multi-Objective Reference-Aligned Machine Unlearning
- 用参考分布约束遗忘目标，防止模型偏离原始知识
- 相比完整重训，删样后性能差距最小
- 适合需要精确删除特定数据的场景
机器删样旨在移除特定训练样本的影响，同时保持模型性能。现有单目标方法如梯度上升或随机重标签常因优化冲突和无界遗忘目标导致灾难性遗忘，使模型偏离预训练知识。我们提出参考对齐删样（RAUL），通过联合优化遗忘与保留，将无界损失最大化替换为对遗忘样本预测与代表未见数据的参考分布之间的有界KL对齐，该参考分布可为均匀分布或来自保留集的实证分布，从而约束遗忘目标并减少与保留目标的梯度冲突。采用雅可比下降求解多目标优化问题，聚合多梯度方向以避免冲突。实验表明，RAUL在删样后性能上最接近完整重训结果。
原文摘要 · Abstract (English)
Machine unlearning aims to remove the influence of specific training samples while preserving the model's utility. Existing single-objective approaches, such as gradient ascent or random relabeling, often induce catastrophic forgetting due to conflicting optimization dynamics and unbounded forgetting objectives that cause the model to drift from its pre-trained knowledge. We propose Reference-Aligned UnLearning (RAUL), a multi-objective framework that jointly optimizes forgetting and retention by replacing unbounded loss maximization with a bounded KL alignment of predictions on forgotten samples toward a reference distribution representing unseen data, instantiated either as a uniform distribution or an empirical distribution from a held-out reference set, which constrains the forgetting objective and reduces gradient conflict with retention. The resulting multi-objective optimization (MOO) problem is solved via Jacobian descent, which aggregates multiple gradients into a direction that does not conflict. Our results demonstrate that RAUL achieves the closest gap compared to full retraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。