用脉冲神经网络实现线性时间图变换,大幅加速大图节点分类。
GT-SNT: A Linear-Time Transformer for Large-Scale Graphs via Spiking Node Tokenization
- 通过脉冲神经网络生成局部感知的脉冲计数嵌入作为节点令牌
- 在多个数据集上性能接近主流模型,推理速度提升最高130倍
- 适合处理大规模图数据,尤其适用于低延迟场景
图注意力网络(GTs)结合消息传递与自注意力机制,在图预测任务中表现优异。然而,可扩展且拓扑感知的节点编码设计滞后于其他模态。随着全注意力机制的二次复杂度,其在大规模图上变得不实用。近期,脉冲神经网络(SNNs)作为类脑模型,可通过事件驱动的脉冲神经元将输入强度转化为离散脉冲表示,具备节能优势。受此启发,我们提出一种基于脉冲节点编码的线性时间图变换模型(GT-SNT),用于节点分类。通过多步特征传播与SNN结合,脉冲节点编码生成紧凑、局部感知的脉冲计数嵌入作为节点令牌,避免预定义码本及其使用问题。码本引导的自注意力机制利用这些令牌实现节点到令牌的关注,完成线性时间的全局上下文聚合。实验表明,GT-SNT在从小到大的节点分类数据集上均达到与现有最优模型相当的性能,推理速度相比其他图变换模型最高提升130倍。
原文摘要 · Abstract (English)
Graph Transformers (GTs), which integrate message passing and self-attention mechanisms simultaneously, have achieved promising empirical results in graph prediction tasks. However, the design of scalable and topology-aware node tokenization has lagged behind other modalities. This gap becomes critical as the quadratic complexity of full attention renders them impractical on large-scale graphs. Recently, Spiking Neural Networks (SNNs), as brain-inspired models, provided an energy-saving scheme to convert input intensity into discrete spike-based representations through event-driven spiking neurons. Inspired by these characteristics, we propose a linear-time Graph Transformer with Spiking Node Tokenization (GT-SNT) for node classification. By integrating multi-step feature propagation with SNNs, spiking node tokenization generates compact, locality-aware spike count embeddings as node tokens to avoid predefined codebooks and their utilization issues. The codebook guided self-attention leverages these tokens to perform node-to-token attention for linear-time global context aggregation. In experiments, we compare GT-SNT with other state-of-the-art baselines on node classification datasets ranging from small to large. Experimental results show that GT-SNT achieves comparable performances on most datasets and reaches up to 130x faster inference speed compared to other GTs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。