解决模型遗忘时对特定群体的不公平影响,提升公平性。
Group-robust Machine Unlearning
- 通过样本重加权实现精确遗忘,缓解主导群体性能下降。
- 提出MIU方法,降低模型特征与群体信息的互信息,减少遗忘偏差。
- 适用于需要公平性保障的机器学习场景,如医疗、金融领域。
机器遗忘是一种新兴范式,旨在移除特定训练数据(即遗忘集)对模型的影响,同时保留其余数据(即保留集)的知识。以往方法假设遗忘数据在所有训练样本中均匀分布,但若遗忘数据在某一群体中占主导(如种族、性别),我们实证发现该群体性能会显著下降,引发公平性问题。本文针对非均匀分布的遗忘集,提出群组鲁棒机器遗忘问题,并给出一种简单有效的精确遗忘策略,通过样本分布重加权缓解主导群体的性能损失。此外,提出首个近似遗忘中的群组鲁棒方法MIU(互信息感知机器遗忘),通过最小化模型特征与群体信息的互信息,在实现遗忘的同时减少主导群体的性能退化。MIU还结合原始模型进行样本分布重加权与互信息校准,以保持群组鲁棒性。在三个数据集上的实验表明,MIU优于标准方法,在实现遗忘的同时不牺牲模型鲁棒性。源代码见:https://github.com/tdemin16/group-robust_machine_unlearning
原文摘要 · Abstract (English)
Machine unlearning is an emerging paradigm to remove the influence of specific training data (i.e., the forget set) from a model while preserving its knowledge of the rest of the data (i.e., the retain set). Previous approaches assume the forget data to be uniformly distributed from all training datapoints. However, if the data to unlearn is dominant in one group (e.g., ethnicity, gender), we empirically show that performance for this group degrades, leading to fairness issues. To perform unlearning while preserving fairness, this work addresses the overlooked problem of non-uniformly distributed forget sets, which we refer to as group-robust machine unlearning. We formalize the problem and present a simple and effective exact unlearning strategy that mitigates the performance loss in dominant groups via sample distribution reweighting. Moreover, we present MIU (Mutual Information-aware Machine Unlearning), the first approach for group robustness in approximate machine unlearning. MIU minimizes the mutual information between model features and group information, achieving unlearning while reducing performance degradation in the dominant group of the forget set. Additionally, MIU exploits sample distribution reweighting and mutual information calibration with the original model to preserve group robustness. We conduct experiments on three datasets and show that MIU outperforms standard methods, achieving unlearning without compromising model robustness. Source code available at https://github.com/tdemin16/group-robust_machine_unlearning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。