arXiv:2602.13106cs.LGcs.AI2026-02被引 1

提出理论框架,揭示GNN能学哪些算法及为何能泛化。

Which Algorithms Can Graph Neural Networks Learn?

  • 构建理论框架,分析MPNN学习算法的充分条件。
  • 证明标准GNN无法学习某些算法,但可设计更强大变体。
  • 对贝尔曼-福特算法优化,减少训练数据需求且支持可微正则化。

近年来,神经架构学习执行离散算法的能力受到广泛关注,这一方向常被称为神经算法推理。目标是将算法推理能力整合进更大规模的神经流水线中。许多此类架构基于图神经网络(MPNN),因其具备置换等变性以及处理稀疏性和可变输入大小的能力。然而,现有研究要么主要依赖经验性结果且缺乏形式化保证,要么仅关注表达能力,未解决模型在有限训练集外如何泛化的问题。本文提出一个通用理论框架,刻画了MPNN从小型实例训练集中学习算法并严格逼近任意规模输入行为的充分条件。该框架适用于广泛算法,包括单源最短路径、最小生成树及动态规划问题(如0-1背包问题)。此外,我们建立了多类任务的不可学习性结果,表明标准MPNN无法学习这些任务,并推导出更具表达力的类似MPNN架构以克服限制。最后,针对贝尔曼-福特算法进行细化分析,显著减少所需训练集规模,并通过引入可微正则化损失扩展了近期工作(Nerem et al., 2025)。实验结果基本验证了理论发现。

原文摘要 · Abstract (English)

In recent years, there has been growing interest in understanding neural architectures' ability to learn to execute discrete algorithms, a line of work often referred to as neural algorithmic reasoning. The goal is to integrate algorithmic reasoning capabilities into larger neural pipelines. Many such architectures are based on (message-passing) graph neural networks (MPNNs), owing to their permutation equivariance and ability to deal with sparsity and variable-sized inputs. However, existing work is either largely empirical and lacks formal guarantees or it focuses solely on expressivity, leaving open the question of when and how such architectures generalize beyond a finite training set. In this work, we propose a general theoretical framework that characterizes the sufficient conditions under which MPNNs can learn an algorithm from a training set of small instances and provably approximate its behavior on inputs of arbitrary size. Our framework applies to a broad class of algorithms, including single-source shortest paths, minimum spanning trees, and general dynamic programming problems, such as the $0$-$1$ knapsack problem. In addition, we establish impossibility results for a wide range of algorithmic tasks, showing that standard MPNNs cannot learn them, and we derive more expressive MPNN-like architectures that overcome these limitations. Finally, we refine our analysis for the Bellman-Ford algorithm, yielding a substantially smaller required training set and significantly extending the recent work of Nerem et al. [2025] by allowing for a differentiable regularization loss. Empirical results largely support our theoretical findings.

图神经网络算法学习理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。