通过特征空间元学习,让模型在分布偏移下更抗虚假相关性。
HSFM: Hard-Set-Guided Feature-Space Meta-Learning for Robust Classification under Spurious Correlations
- 在特征空间直接做数据增强,优化分类头性能
- 冻结主干网络,仅训练轻量分类头,提升少数群体表现
- 方法高效稳定,单卡几分钟即可完成训练
深度神经网络常依赖虚假特征进行预测,导致在分布偏移或少数群体样本上表现脆弱。近期研究发现,经验风险最小化(ERM)训练的模型主干虽能提取丰富表征,但分类头是性能下降的主要原因。若冻结主干并重训轻量分类头,可在分布偏移和少数群体上显著提升性能。受此启发,我们提出一种双层元学习方法,直接在特征空间进行增强,以改进分类头对虚假相关性的处理。该方法学习支持集的特征修改,使得在少量内循环更新后,分类器在困难样本上的损失更低,最差组性能提升。相比像素空间或端到端优化,本方法在主干输出层操作,效率高且稳定,单卡仅需几分钟训练。进一步通过CLIP可视化验证,所学特征空间更新能引发语义上有意义的偏移,与虚假属性对齐。
原文摘要 · Abstract (English)
Deep neural networks often rely on spurious features to make predictions, which makes them brittle under distribution shift and on samples where the spurious correlation does not hold (e.g., minority-group examples). Recent studies have shown that, even in such settings, the feature extractor of an Empirical Risk Minimization (ERM)-trained model can learn rich and informative representations, and that much of the failure may be attributed to the classifier head. In particular, retraining a lightweight head while keeping the backbone frozen can substantially improve performance on shifted distributions and minority groups. Motivated by this observation, we propose a bilevel meta-learning method that performs augmentation directly in feature space to improve spurious correlation handling in the classifier head. Our method learns support-side feature edits such that, after a small number of inner-loop updates on the edited features, the classifier achieves lower loss on hard examples and improved worst-group performance. By operating at the backbone output rather than in pixel space or through end-to-end optimization, the method is highly efficient and stable, requiring only a few minutes of training on a single GPU. We further validate our method with CLIP-based visualizations, showing that the learned feature-space updates induce semantically meaningful shifts aligned with spurious attributes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。