训练种子少会误导模型遗忘评估结果，需多组训练种子才可靠。
On the importance of multiple training seeds for evaluating machine unlearning
- 用同一训练种子多次运行遗忘算法，结果不可靠。
- 不同训练种子导致遗忘性能差异显著，尤其在确定性方法中。
- 建议同时增加训练和遗忘种子数，提升评估可信度。
机器遗忘旨在无需重新训练即可移除模型中特定数据点的影响。大多数实用的遗忘算法仅为近似方法，其性能只能通过实验评估。当前常见做法是使用同一训练模型、多次独立运行（即多个遗忘种子）来测试遗忘效果。然而，在图像分类实验中，该方法可能产生非代表性结果，因为遗忘表现对训练种子的选择敏感。这在确定性遗忘方法中尤为明显，因其从相同模型出发总会得到一致结果。联邦学习排序与大语言模型的进一步实验表明，此问题不仅限于图像分类。我们还解释了为何增加遗忘种子数无法弥补单一训练种子的缺陷。最后，我们提供如何选择训练和遗忘种子数量的指导。
原文摘要 · Abstract (English)
Machine unlearning aims to remove the influence of certain data points from a trained model without costly retraining. Most practical unlearning algorithms are only approximate and their performance can only be assessed empirically. Common practice is to run unlearning algorithms multiple times independently (i.e., using multiple unlearning seeds) starting from the same trained model (i.e., using only a single training seed ). In image-classification experiments, this practice can give non-representative results as unlearning performance can be sensitive to the choice of training seed. This is particularly relevant for deterministic unlearning methods which always produce the same result when started from the same trained model. Further experiments on federated learning-to-rank, and large language models confirm that this issue extends beyond image classification. We also explain why increasing the number of unlearning seeds cannot generally compensate for the lack of multiple training seeds. Finally, we give guidance on how to select the number of training and unlearning seeds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。