arXiv:2605.08730cs.LGcs.CR2026-05

发现分类头偏置偏差会误导模型遗忘,提出新方法让遗忘更可靠。

Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation

论文配图:Classification-Head Bias in Class-Level Machine Unlearning: Diagnosis, Mitigation, and Evaluation
图 1 · 摘自论文原文
  • 通过调节分类头偏置实现快速遗忘,揭示内部机制缺陷
  • 在多个数据集上验证,新方法保持高准确率且偏置更稳定
  • 适合关注模型可解释性与安全性的研究人员

类级机器遗忘旨在移除特定类别的影响,同时保留对保留类别的模型性能。现有方法通常以保留集准确率、遗忘集准确率和遗忘时间评估,但这些指标难以反映遗忘的内部实现机制。本文揭示类级遗忘中存在偏置主导的捷径:通过减小分类头中对应类别的偏置项,即可抑制被遗忘类别的预测。我们分析了软最大交叉熵训练下分类头偏置的梯度动态,解释为何仅优化保留集会导致缺失类别偏置下降。基于此,提出BiasShift诊断基线,证明简单偏置调整可满足传统指标,却留下异常偏置模式暴露遗忘标签。为缓解过度抑制,提出两种偏置感知机制:两阶段偏置梯度反转(TS-BGRM)和下界铰链正则化(LB-HR)。进一步引入三项偏置导向指标:偏置稳定性系数(BSC)、中位偏置差(MBG)和最小偏置得分(MBS),用于量化偏置依赖与潜在泄漏。在CIFAR-10、CIFAR-100和Tiny-ImageNet上的实验表明,所提方法在保持竞争力遗忘性能的同时,产生更稳定的偏置分布。代码已公开于 {https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redistribution}。

原文摘要 · Abstract (English)

Class-level machine unlearning aims to remove the influence of specified classes while preserving model utility on retained classes. Existing methods are commonly evaluated by retain-set accuracy, forget-set accuracy, and unlearning time, but these metrics provide limited insight into how forgetting is achieved internally. In this paper, we reveal a bias-dominated shortcut in class-level unlearning: the prediction of forgotten classes can be suppressed by decreasing the corresponding bias terms in the final classification head. We first analyze the gradient dynamics of classification-head biases under softmax cross-entropy training, explaining why retain-set-only optimization tends to reduce the biases of absent classes. Based on this observation, we introduce BiasShift as a diagnostic baseline, showing that simple bias manipulation can satisfy conventional unlearning metrics while leaving abnormal bias patterns that reveal forgotten labels. To mitigate excessive forgotten-class bias suppression, we propose two bias-aware mechanisms, namely Two-Stage Bias Gradient Reversal Mechanism (TS-BGRM) and Lower-Bound Hinge Regularization (LB-HR). We further introduce three bias-oriented metrics, including Bias Stability Coefficient (BSC), Median Bias Gap (MBG), and Minimal Bias Score (MBS), to quantify bias dependence and potential leakage. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet demonstrate that the proposed methods maintain competitive unlearning performance while producing more stable bias distributions. We have released our code at {https://github.com/zwd2024/Beyond-the-Shadow-of-Bias-From-Classification-Head-Bias-to-Parameter-Redistribution}.

机器遗忘偏置分析模型安全可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。