通过正则化特征表示，缓解模型对虚假相关性的依赖，提升少数群体的鲁棒性。
Elastic Representation: Mitigating Spurious Correlations for Group Robustness
- 在最后一层特征上施加核范数与弗罗贝尼乌斯范数惩罚，平衡重要特征与多样性。
- 在有限或不平衡数据下显著提升少数群体的预测准确率，同时不损害整体性能。
- 方法简单通用，可嵌入多种深度学习模型，适合关注公平性和鲁棒性的研究者。
深度学习模型在依赖输入特征与标签之间的虚假相关性时，可能出现严重性能下降，尤其在训练数据有限或不平衡时更为明显。尽管已有大量工作聚焦于学习与标签具有一致相关性的不变特征，却忽略了特征间虚假相关性的潜在危害。本文提出弹性表示（ElRep），通过对神经网络最后一层的表示施加核范数和弗罗贝尼乌斯范数惩罚，实现重要特征保留与特征多样性的平衡。该方法类似弹性网络，兼具高效性与鲁棒性。实验表明，该方法能有效缓解虚假相关性，提升少数群体的预测表现，且理论上证明其对分布内预测的影响最小。这一优势显著优于以牺牲整体性能为代价优化少数群体的方法。
原文摘要 · Abstract (English)
Deep learning models can suffer from severe performance degradation when relying on spurious correlations between input features and labels, making the models perform well on training data but have poor prediction accuracy for minority groups. This problem arises especially when training data are limited or imbalanced. While most prior work focuses on learning invariant features (with consistent correlations to y), it overlooks the potential harm of spurious correlations between features. We hereby propose Elastic Representation (ElRep) to learn features by imposing Nuclear- and Frobenius-norm penalties on the representation from the last layer of a neural network. Similar to the elastic net, ElRep enjoys the benefits of learning important features without losing feature diversity. The proposed method is simple yet effective. It can be integrated into many deep learning approaches to mitigate spurious correlations and improve group robustness. Moreover, we theoretically show that ElRep has minimum negative impacts on in-distribution predictions. This is a remarkable advantage over approaches that prioritize minority groups at the cost of overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。