无需标注数据，通过自蒸馏让模型自动消除偏差
Debiasify: Self-Distillation for Unsupervised Bias Mitigation
- 用自蒸馏把深层复杂特征迁移到浅层，强制模型忽略表面相关性
- 在CelebA上使最差群体准确率提升10.13%，超越现有无监督方法
- 适合处理缺乏标注的场景，如医疗影像、隐私敏感领域
简化偏差是神经网络中的重大挑战，常导致模型偏好简单解并意外学习到由虚假相关性引发的决策规则，从而产生泛化能力下降的偏见模型。现有许多方法依赖人工标注，但获取各类偏差属性的标注往往不切实际。为此，我们提出Debiasify，一种无需预先了解偏差性质的新型自蒸馏方法。该方法利用新的蒸馏损失，以无监督方式将网络中深层包含复杂高预测性特征的知识，传递至浅层具有简单属性条件特征的层。这使得Debiasify能够学习鲁棒且去偏的表征，在多种偏差和数据集上表现出良好的泛化能力，同时提升最差群体性能与整体准确率。在计算机视觉和医学影像基准上的大量实验表明，该方法显著优于以往无监督去偏方法（如在CelebA的波浪发分类任务中，最差群体准确率提升10.13%），并达到或超过有监督方法的性能。代码已公开。
原文摘要 · Abstract (English)
Simplicity bias poses a significant challenge in neural networks, often leading models to favor simpler solutions and inadvertently learn decision rules influenced by spurious correlations. This results in biased models with diminished generalizability. While many current approaches depend on human supervision, obtaining annotations for various bias attributes is often impractical. To address this, we introduce Debiasify, a novel self-distillation approach that requires no prior knowledge about the nature of biases. Our method leverages a new distillation loss to transfer knowledge within the network, from deeper layers containing complex, highly-predictive features to shallower layers with simpler, attribute-conditioned features in an unsupervised manner. This enables Debiasify to learn robust, debiased representations that generalize effectively across diverse biases and datasets, improving both worst-group performance and overall accuracy. Extensive experiments on computer vision and medical imaging benchmarks demonstrate the effectiveness of our approach, significantly outperforming previous unsupervised debiasing methods (e.g., a 10.13% improvement in worst-group accuracy for Wavy Hair classification in CelebA) and achieving comparable or superior performance to supervised approaches. Our code is publicly available at the following link: Debiasify.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。