用全层特征重加权，提升模型在少数群体上的准确率
Not Only the Last-Layer Features for Spurious Correlations: All Layer Deep Feature Reweighting
- 从所有网络层提取特征，而非仅最后一层
- 在多个基准数据集上显著提升最差群体准确率
- 适合关注公平性与鲁棒性的模型开发者
虚假相关性是机器学习模型的重要错误来源，尤其在追求群体公平性时。近期研究表明，通过在平衡验证集上重新训练最后一层，可隔离出对预测有贡献的稳健特征。然而，神经网络在向最后一层传递过程中可能丢弃关键属性。本文提出从所有网络层提取特征并进行重训练，结合一种新提出的特征选择策略，筛选出无偏特征。实验表明，该方法在多个标准基准上显著提升了最差群体的准确率。
原文摘要 · Abstract (English)
Spurious correlations are a major source of errors for machine learning models, in particular when aiming for group-level fairness. It has been recently shown that a powerful approach to combat spurious correlations is to re-train the last layer on a balanced validation dataset, isolating robust features for the predictor. However, key attributes can sometimes be discarded by neural networks towards the last layer. In this work, we thus consider retraining a classifier on a set of features derived from all layers. We utilize a recently proposed feature selection strategy to select unbiased features from all the layers. We observe this approach gives significant improvements in worst-group accuracy on several standard benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。