通过噪声反事实配对提升模型对虚假相关性的鲁棒性
From Invariant Representations to Invariant Data: Provable Robustness to Spurious Correlations via Noisy Counterfactual Matching
- 用反事实样本对构建数据级不变性，而非依赖特征表示
- 仅需少量带噪反事实对即显著降低测试域误差
- 理论证明误差受反事实质量与多样性影响，适合数据稀缺场景
从训练数据中学习虚假相关性的模型在新环境中常失效。尽管许多方法致力于学习不变表示以解决此问题，但其性能往往不及标准经验风险最小化（ERM）。本文提出一种以数据为中心的替代方案，将关注点从学习不变表示转向利用不变数据对——即应具有相同预测结果的样本对。我们证明某些反事实天然具备这种不变性。基于此，提出噪声反事实匹配（NCM）方法，一种基于约束的简单策略，通过利用少量带噪反事实对提升鲁棒性，优于此前未显式考虑噪声的方法。对于线性因果模型，我们证明了NCM在测试域的误差被其域内误差加上一个依赖反事实质量与多样性的项所限定。合成数据实验验证了理论，真实数据集上的表现也证明了该方法的有效性。
原文摘要 · Abstract (English)
Models that learn spurious correlations from training data often fail when deployed in new environments. While many methods aim to learn invariant representations to address this, they often underperform standard empirical risk minimization (ERM). We propose a data-centric alternative that shifts the focus from learning invariant representations to leveraging invariant data pairs -- pairs of samples that should have the same prediction. We prove that certain counterfactuals naturally satisfy this invariance property. Based on this, we introduce Noisy Counterfactual Matching (NCM), a simple constraint-based method that improves robustness by leveraging even a small number of \emph{noisy} counterfactual pairs -- improving upon prior works that do not explicitly consider noise. For linear causal models, we prove that NCM's test-domain error is bounded by its in-domain error plus a term dependent on the counterfactuals' quality and diversity. Experiments on synthetic data validate our theory, and we demonstrate NCM's effectiveness on real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。