通过分层抽象与递归机制,提升图模型对远距离依赖的建模能力。
Graph Hierarchical Recurrence for Long-Range Generalization

- 在原始图与分层抽象图上联合递归计算,捕捉长程关系。
- 在长距离任务上性能超越现有模型,参数量仅为1%。
- 适合需要高效泛化、资源受限的图学习场景。
图神经网络(GNN)和图变压器(GT)已成为图学习的核心范式,结合了深度模型的表征学习能力与归纳偏置带来的样本效率。尽管表现优异,大量研究表明这些模型在捕捉图中远距离区域相关性方面仍存在根本性局限。针对此问题,本文提出图层次递归(GHR)框架,同时在输入图及其通过池化获得的层次抽象上进行运算。我们还发现,现有模型在分布外泛化任务中的缺陷更为显著,即测试实例涉及的交互距离超出训练时观察范围。相比之下,尽管设计简单,GHR具备三大优势:强长程依赖建模能力、改进的分布外泛化性能以及高参数效率。在一系列长程基准测试中,GHR始终优于现有图模型,且参数量仅需当前最先进模型的1%。结果表明,单纯扩大模型容量可能不足以实现有效泛化,提示应探索与当前架构扩展趋势互补的新方向。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) and Graph Transformers (GTs) are now a fundamental paradigm for graph learning, combining the representation-learning capabilities of deep models with the sample efficiency induced by their inductive biases. Despite their effectiveness, a large body of work has shown that these models still face fundamental limitations in tasks that require capturing correlations between distant regions of a graph. To address this issue, we introduce Graph Hierarchical Recurrence (GHR), a novel framework that operates jointly on the input graph and on a hierarchical abstraction obtained through pooling. We also show that the limitations of existing models are even more pronounced in out-of-range generalization, where test instances involve interactions over distances longer than those observed during training. By contrast, despite its simple design, GHR provides three key advantages: strong performance on long-range dependencies, improved out-of-range generalization, and high parameter efficiency. To corroborate these claims, we show that across a broad set of long-range benchmarks, GHR consistently outperforms existing graph models while using as little as 1% of the parameters of current state-of-the-art models. These results suggest a complementary direction to the current trend of scaling architectures to obtain graph foundation models, indicating that increased model capacity alone may not be sufficient for generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。