隐藏混杂因素导致分布偏移,使因果不变表示失效。
When Shift Happens - Confounding Is to Blame
- 发现隐藏混杂变化是分布偏移的根源
- 传统方法在混合变量下表现反而更好
- 需建模环境特异性关系提升鲁棒性
分布偏移引入不确定性,削弱机器学习模型的鲁棒性和泛化能力。尽管主流观点认为学习因果不变表征可增强对分布偏移的鲁棒性,但近期实证研究揭示反直觉现象:(i) 经典经验风险最小化(ERM)可媲美甚至超越最先进的分布外(OOD)泛化方法;(ii) 当使用所有可用协变量而非仅因果协变量时,其OOD泛化性能进一步提升。基于实证与理论证据,我们归因于隐藏混杂。隐藏混杂的改变引发数据分布变化,违反现有OOD泛化方法的常见假设。在此条件下,我们证明有效泛化需学习环境特异性关系,而非依赖不变关系。此外,我们展示通过引入隐藏混杂代理变量可缓解此类挑战。这些发现为设计鲁棒的OOD泛化算法及合理协变量选择策略提供新的理论洞见与实践指导。
原文摘要 · Abstract (English)
Distribution shifts introduce uncertainty that undermines the robustness and generalization capabilities of machine learning models. While conventional wisdom suggests that learning causal-invariant representations enhances robustness to such shifts, recent empirical studies present a counterintuitive finding: (i) empirical risk minimization (ERM) can rival or even outperform state-of-the-art out-of-distribution (OOD) generalization methods, and (ii) its OOD generalization performance improves when all available covariates, not just causal ones, are utilized. Drawing on both empirical and theoretical evidence, we attribute this phenomenon to hidden confounding. Shifts in hidden confounding induce changes in data distributions that violate assumptions commonly made by existing OOD generalization approaches. Under such conditions, we prove that effective generalization requires learning environment-specific relationships, rather than relying solely on invariant ones. Furthermore, we show that models augmented with proxies for hidden confounders can mitigate the challenges posed by hidden confounding shifts. These findings offer new theoretical insights and practical guidance for designing robust OOD generalization algorithms and principled covariate selection strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。