用低秩分块图高效表示大图,让神经网络训练快又省内存。
Efficient Learning on Large Graphs using a Densifying Regularity Lemma
- 将大图拆成相交的双社区块,用低秩结构近似原图
- 无论图多稀疏,都能用固定秩的稠密块图逼近,精度可控
- 模型内存和计算量只随节点数线性增长,适合超大规模图
在大规模图上学习面临巨大挑战,传统消息传递神经网络的计算与内存开销随边数线性增长。本文提出相交块图(Intersecting Block Graph, IBG),一种基于相交二分组件的低秩分解方法,每个组件由源节点和目标节点的两个社区构成。通过降低非边权重,可高效近似任意稀疏或稠密图。我们证明了弱正则性引理的一个构造性版本:对任意给定精度,任意规模或稀疏度的图均可被一个仅依赖精度的稠密IBG近似,其秩与稀疏度无关。这与以往形式形成对比。我们设计了一种在IBG表示上运行的图神经网络架构,在节点分类、时空图分析和知识图谱补全任务中表现优异,且内存与计算复杂度仅随节点数线性增长,而非边数。
原文摘要 · Abstract (English)
Learning on large graphs presents significant challenges, with traditional Message Passing Neural Networks suffering from computational and memory costs scaling linearly with the number of edges. We introduce the Intersecting Block Graph (IBG), a low-rank factorization of large directed graphs based on combinations of intersecting bipartite components, each consisting of a pair of communities, for source and target nodes. By giving less weight to non-edges, we show how to efficiently approximate any graph, sparse or dense, by a dense IBG. Specifically, we prove a constructive version of the weak regularity lemma, showing that for any chosen accuracy, every graph, regardless of its size or sparsity, can be approximated by a dense IBG whose rank depends only on the accuracy. This dependence of the rank solely on the accuracy, and not on the sparsity level, is in contrast to previous forms of the weak regularity lemma. We present a graph neural network architecture operating on the IBG representation of the graph and demonstrating competitive performance on node classification, spatio-temporal graph analysis, and knowledge graph completion, while having memory and computational complexity linear in the number of nodes rather than edges.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。