通过生成变体与自监督学习，让模型摆脱背景干扰，提升分布外泛化能力。
Breaking Spurious Correlations via Generative Randomization and Cross-Variant Self-Supervised Learning
- 用扩散模型生成同一物体不同背景的变体图像，保留物体身份
- 设计对比学习机制，使模型学会忽略背景差异，聚焦物体本身
- 在多个分布外数据集上表现优异，尤其在小群体上性能领先
基于经验风险最小化的深度神经网络在分布偏移下常因依赖对象标签与背景之间的虚假相关而失效。现有生成方法虽能生成反事实图像，但通常仅作为常规增强，模型仍可能保留对背景敏感的表征。本文提出两阶段框架：首先使用零样本分割提取前景，并通过结构保持型扩散模型生成背景变化的图像变体；其次引入跨变体自监督学习，将同一物体在不同背景下的变体作为正样本对，在对比目标下促使编码器对齐以物体为中心的表征，抑制背景特定线索。最后，采用分层学习率的GroupDRO微调预训练编码器。在多个分布外基准测试中表现最佳，水鸟数据集达到92.5%，MetaShift为81.7%，NICO++为87.4%。代码已开源。
原文摘要 · Abstract (English)
Deep neural networks trained with Empirical Risk Minimization (ERM) often fail under distribution shifts because they exploit spurious correlations between object labels and background context. Recent generative approaches address this issue by creating counterfactual images with altered contexts, but typically use these samples as standard data augmentation, leaving the model free to retain background-sensitive representations. We propose a two-stage framework that uses generative intervention to explicitly learn background-invariant visual representations. First, we isolate the foreground object using zero-shot segmentation and generate context-shifted variants with a structure-preserving diffusion model, preserving object identity while varying the surrounding environment. We then introduce Cross-Variant Self-Supervised Learning, where variants of the same object under different backgrounds form positive pairs in a contrastive objective. This encourages the encoder to align object-centric representations while suppressing background-specific cues. Then, we fine-tune the pretrained encoder using an ERM warm-up followed by GroupDRO with layer-wise learning rates. Experiments on distribution-shift benchmarks demonstrate best worst-group performance, achieving 92.5% on Waterbirds, 81.7% on MetaShift, and 87.4% on NICO++. Code: https://github.com/surajyadav-research/GRSSL
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。