通过定向丢弃记忆少数群体样本的神经元，提升模型对少数群体的泛化能力。
FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups
- 引入例相关丢弃（FairDropout），将少数群体样本的记忆定位到特定神经元并推理时丢弃。
- 在视觉、语言和医疗任务上显著降低对伪相关特征的依赖，优于现有方法。
- 适合关注公平性、少数群体表现提升的模型开发者与研究人员。
深度学习模型常利用训练数据中的伪特征以实现低训练误差，但在测试分布偏移时泛化能力差。为解决此问题，已有多种来自不平衡学习、表示学习和分类器校准的方法被提出。本文观察到，基于经验风险最小化的模型对多数群体泛化良好，却会记忆少数群体的样本实例。基于近期发现——记忆可局限于少数神经元，我们提出例相关丢弃（FairDropout），将此类记忆导向特定神经元，并在推理时将其丢弃。我们在涵盖视觉、语言和医疗任务的子群体基准套件上进行实证评估，结果表明该方法显著减少对伪相关特征的依赖，且性能优于当前最优方法。
原文摘要 · Abstract (English)
Deep learning models frequently exploit spurious features in training data to achieve low training error, often resulting in poor generalization when faced with shifted testing distributions. To address this issue, various methods from imbalanced learning, representation learning, and classifier recalibration have been proposed to enhance the robustness of deep neural networks against spurious correlations. In this paper, we observe that models trained with empirical risk minimization tend to generalize well for examples from the majority groups while memorizing instances from minority groups. Building on recent findings that show memorization can be localized to a limited number of neurons, we apply example-tied dropout as a method we term FairDropout, aimed at redirecting this memorization to specific neurons that we subsequently drop out during inference. We empirically evaluate FairDropout using the subpopulation benchmark suite encompassing vision, language, and healthcare tasks, demonstrating that it significantly reduces reliance on spurious correlations, and outperforms state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。