提出GOODFormer,让图Transformer在分布外数据上更鲁棒。
Invariant Graph Transformer for Out-of-Distribution Generalization
- 通过分离不变与可变子图,保持注意力聚焦关键结构。
- 动态子图编码器高效捕捉变化中的图结构特征。
- 适合需要跨数据集泛化的图学习任务,如生物网络分析。
图变压器(GTs)在各类图分析任务中表现优异,但现有方法主要针对同分布的训练与测试数据,难以应对分布偏移。图不变性学习旨在捕捉标签下具有普遍性的图结构模式,是潜在解决方案,但如何基于不变性原则设计注意力机制和位置/结构编码(PSE)仍具挑战。为此,本文提出图分布外泛化变压器(GOODFormer),通过联合优化三个模块,学习通用图表示。首先,设计基于熵引导的图变压器不变子图解耦器,分离不变与可变子图,同时保持注意力函数的锐度。其次,提出动态子图位置与结构编码器,有效高效地捕捉训练过程中动态变化的子图编码信息。最后,构建不变性学习模块,利用子图节点表示与编码生成可泛化至未见图的图表示。本文还提供了理论支持。在基准数据集上的大量实验表明,该方法在分布偏移下显著优于当前最优基线。
原文摘要 · Abstract (English)
Graph Transformers (GTs) have demonstrated great effectiveness across various graph analytical tasks. However, the existing GTs focus on training and testing graph data originated from the same distribution, but fail to generalize under distribution shifts. Graph invariant learning, aiming to capture generalizable graph structural patterns with labels under distribution shifts, is potentially a promising solution, but how to design attention mechanisms and positional and structural encodings (PSEs) based on graph invariant learning principles remains challenging. To solve these challenges, we introduce Graph Out-Of-Distribution generalized Transformer (GOODFormer), aiming to learn generalized graph representations by capturing invariant relationships between predictive graph structures and labels through jointly optimizing three modules. Specifically, we first develop a GT-based entropy-guided invariant subgraph disentangler to separate invariant and variant subgraphs while preserving the sharpness of the attention function. Next, we design an evolving subgraph positional and structural encoder to effectively and efficiently capture the encoding information of dynamically changing subgraphs during training. Finally, we propose an invariant learning module utilizing subgraph node representations and encodings to derive generalizable graph representations that can to unseen graphs. We also provide theoretical justifications for our method. Extensive experiments on benchmark datasets demonstrate the superiority of our method over state-of-the-art baselines under distribution shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。