arXiv:2605.20732cs.CV2026-05

用注意力机制替代池化层,让模型更好区分核心与虚假特征。

Deep Attention Reweighting: Post-Hoc Attention-Based Feature Aggregation in CNNs for Disentangling Core and Spurious Features under Spurious Correlations

论文配图:Deep Attention Reweighting: Post-Hoc Attention-Based Feature Aggregation in CNNs for Disentangling Core and Spurious Features under Spurious Correlations
图 1 · 摘自论文原文
  • 引入可学习的注意力加权机制,动态抑制空间中虚假特征
  • 在多个数据集上显著降低对虚假相关性的依赖,提升泛化性能
  • 适合关注模型公平性与鲁棒性的研究人员使用

卷积神经网络常利用数据集中的虚假相关性,学习到表面预测但无因果意义的特征,导致泛化能力差和公平性问题。深度特征重加权(DFR)是一种后处理技术,通过在目标数据集上重新训练分类头来减少模型对虚假相关性的依赖。然而,我们发现DFR受限于对纠缠特征的操作,难以同时增强核心特征并抑制虚假特征。我们追溯这种纠缠源于广泛使用的全局平均池化(GAP)层,它将空间上分离的核心与虚假特征混合为单一表示。为此,我们提出深度注意力重加权(DAR),一种可后处理的基于注意力的特征聚合模块,取代GAP,并与分类头联合重训练。DAR在特征图上计算空间位置的自适应权重,实现在特征融合前选择性抑制虚假特征。在多种数据集、评估指标和消融实验中,DAR始终优于DFR,证明注意力聚合有效缓解了由GAP引发的特征纠缠,降低了对虚假相关性的依赖。

原文摘要 · Abstract (English)

Convolutional Neural Networks (CNNs) often exploit spurious correlations in datasets, learning superficially predictive yet causally irrelevant features, leading to poor generalization and fairness issues. Deep Feature Reweighting (DFR) is a post-hoc technique that reduces a trained model's reliance on spurious correlations by retraining its classification head on a target dataset. However, we show that DFR is fundamentally constrained by operating on entangled features, limiting its ability to amplify the core features while simultaneously suppressing the spurious ones. We trace this entanglement to the ubiquitous Global Average Pooling (GAP) layer, which indiscriminately collapses spatially distinct core and spurious features into a single representation. To address this, we propose Deep Attention Reweighting (DAR), a post-hoc attention-based aggregation module that replaces GAP and is retrained jointly with the classification head. DAR computes an adaptive weighting of spatial locations across feature maps, enabling selective suppression of spurious features before the collapse into entangled features. Across various datasets, metrics, and ablations, DAR consistently outperforms DFR, demonstrating that our attention-based aggregation mitigates GAP-induced entanglement and reduces spurious reliance.

注意力机制模型公平性特征解耦

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。