用少量反事实样本引导模型忽略无关特征，提升鲁棒性。
Invariance Pair Guidance: Robustness to Spurious Correlations via Corrective Gradients
- 通过反事实样本对构建不变性约束，动态修正优化方向。
- 在多个数据集上实现对群体偏移的鲁棒性，效果优于现有方法。
- 仅需稀疏标注，适合缺乏密集标签的现实场景。
机器学习模型易受训练数据分布影响，常依赖非因果捷径。现有方法多需密集分组标签、多域数据或特殊预处理，限制了应用。本文提出不变性配对引导（IPG），利用少量反事实样本对来减少对虚假相关性的依赖。IPG设计一种双更新机制，通过隔离虚假属性的输入对定义不变性，并据此生成修正梯度，动态调整优化路径。该修正基于预设的不变性条件自适应调整。在ColoredMNIST、Waterbirds-100和CelebA上的实验表明，IPG在群体偏移下仍保持鲁棒，且具备理论收敛性支持。该方法数据高效，具有理论基础。
原文摘要 · Abstract (English)
Machine learning models are inherently bound to the distribution of the training data, often exploiting non-causal shortcuts. As a result, achieving robustness to spurious correlations remains a challenge. While existing approaches rely on data manipulation or re-weighting strategies to achieve robustness, they typically require dense group labels, multiple training domains, or specialized pre-processing. We propose Invariance Pair Guidance (IPG), a method to mitigate reliance on spurious correlations using a sparse set of counterfactual pairs. Unlike other methods demanding extensive supervision, IPG utilizes a novel dual-update mechanism to dynamically correct the optimization trajectory. We generate input pairs that isolate the spurious attribute to define the invariance, a characteristic that should not affect the outcome of the model. Based on these pairs, we define a corrective gradient that complements the traditional gradient descent approach. The correction adapts via a predefined invariance condition. Experiments on ColoredMNIST, Waterbirds-100, and CelebA datasets demonstrate the effectiveness of our approach and its robustness to group shifts, supported by a theoretical convergence analysis. IPG offers a data-efficient and theoretically grounded path to robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。