用高层类别指导解耦特征，无需标注就能缓解虚假关联。
Superclass-Guided Representation Disentanglement for Spurious Correlation Mitigation
- 用预训练视觉语言模型的高层类别做特征解耦引导
- 在多个领域泛化任务中显著优于基线方法
- 适合缺乏分组标注的鲁棒性建模场景
为提升对虚假相关性的群体鲁棒性，现有方法通常依赖辅助分组标注，并假设训练与测试域具有相同的分组结构。为克服这些限制，本文提出利用超类——即语义层级高于任务真实标签的类别——作为更内在的信号来识别虚假相关性。模型通过基于梯度的注意力对齐，从预训练视觉语言模型中获取超类指导，再结合理论支持的极小极大最优特征使用策略实现特征解耦。实验表明，该方法在多种领域泛化任务中显著优于强基线，且超越了视觉语言模型本身的指导效果，在定量指标和定性可视化上均有明显提升。
原文摘要 · Abstract (English)
To enhance group robustness to spurious correlations, prior work often relies on auxiliary group annotations and assumes identical sets of groups across training and test domains. To overcome these limitations, we propose to leverage superclasses -- categories that lie higher in the semantic hierarchy than the task's actual labels -- as a more intrinsic signal than group labels for discerning spurious correlations. Our model incorporates superclass guidance from a pretrained vision-language model via gradient-based attention alignment, and then integrates feature disentanglement with a theoretically supported minimax-optimal feature-usage strategy. As a result, our approach attains robustness to more complex group structures and spurious correlations, without the need to annotate any training samples. Experiments across diverse domain generalization tasks show that our method significantly outperforms strong baselines and goes well beyond the vision-language model's guidance, with clear improvements in both quantitative metrics and qualitative visualizations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。