提出新方法提升图数据分布外泛化能力,解决现有方法依赖虚假特征的问题。
Dissecting the Failure of Invariant Learning on Graphs
- 通过结构因果模型分析发现,传统不变学习方法缺乏类别条件不变性约束。
- 提出CIA-LRA方法,在无环境标签时仍能有效分离不变特征与虚假特征。
- 理论证明其泛化误差界,实验验证在多个图数据集上显著优于基线方法。
提升图上节点级分布外(OOD)泛化能力仍是关键研究方向。本文构建结构因果模型(SCM),从理论上剖析两种主流不变学习方法——不变风险最小化(IRM)和方差风险外推(VREx)——在节点级OOD场景下的表现。分析揭示关键缺陷:由于缺乏类别条件不变性约束,这些方法难以准确识别预测的不变自图结构,进而依赖虚假特征。为此,我们提出跨环境类内对齐(CIA),通过条件于同一类别的跨环境表示对齐,显式消除虚假特征,无需事先知晓因果结构。为适应节点级OOD中环境标签难获取的情况,进一步提出局部重加权对齐(CIA-LRA),利用邻近标签分布选择性对齐节点表示,有效区分并保留不变特征,同时去除虚假特征,且不依赖环境标签。我们基于PAC-Bayesian分析推导了CIA-LRA的分布外泛化误差界,理论上证明其有效性。在多个图分布外基准测试上的实验验证了CIA与CIA-LRA的优越性,标志着节点级分布外泛化的重要进展。代码已公开于https://github.com/NOVAglow646/NeurIPS24-Invariant-Learning-on-Graphs。
原文摘要 · Abstract (English)
Enhancing node-level Out-Of-Distribution (OOD) generalization on graphs remains a crucial area of research. In this paper, we develop a Structural Causal Model (SCM) to theoretically dissect the performance of two prominent invariant learning methods -- Invariant Risk Minimization (IRM) and Variance-Risk Extrapolation (VREx) -- in node-level OOD settings. Our analysis reveals a critical limitation: due to the lack of class-conditional invariance constraints, these methods may struggle to accurately identify the structure of the predictive invariant ego-graph and consequently rely on spurious features. To address this, we propose Cross-environment Intra-class Alignment (CIA), which explicitly eliminates spurious features by aligning cross-environment representations conditioned on the same class, bypassing the need for explicit knowledge of the causal pattern structure. To adapt CIA to node-level OOD scenarios where environment labels are hard to obtain, we further propose CIA-LRA (Localized Reweighting Alignment) that leverages the distribution of neighboring labels to selectively align node representations, effectively distinguishing and preserving invariant features while removing spurious ones, all without relying on environment labels. We theoretically prove CIA-LRA's effectiveness by deriving an OOD generalization error bound based on PAC-Bayesian analysis. Experiments on graph OOD benchmarks validate the superiority of CIA and CIA-LRA, marking a significant advancement in node-level OOD generalization. The codes are available at https://github.com/NOVAglow646/NeurIPS24-Invariant-Learning-on-Graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。