arXiv:2511.12442cs.LGcs.AI2025-11AAAI

提出无需注意力机制的动态图模型,高效捕捉长期依赖关系。

Global-Lens Transformers: Adaptive Token Mixing for Dynamic Link Prediction

  • 用自适应令牌混合器替代注意力,按交互顺序与时间间隔进行局部聚合。
  • 在6个基准上达到最优性能,训练效率显著提升。
  • 适合处理高频、大规模动态图,如社交网络与推荐系统。

动态图学习在建模随时间演化的关系中至关重要,尤其在交通系统、社交网络和推荐平台中的时序链接预测任务中。尽管基于Transformer的模型通过捕捉长程时间依赖表现出色,但其对自注意力的依赖导致序列长度的二次复杂度,限制了在高频率或大规模图上的可扩展性。本文重新审视了自注意力在动态图建模中的必要性。受近期研究启发,即Transformer的成功更多源于架构设计而非注意力机制本身,我们提出GLFormer——一种新颖的无注意力Transformer式框架。GLFormer引入自适应令牌混合器,根据交互顺序和时间间隔执行上下文感知的局部聚合;为捕捉长期依赖,进一步设计分层聚合模块,通过堆叠多层局部令牌混合器扩展时间感受野。在六个广泛使用的动态图基准上的实验表明,GLFormer实现当前最优性能,揭示无注意力架构可在动态图场景中媲美甚至超越传统Transformer基线,且效率显著提升。

原文摘要 · Abstract (English)

Dynamic graph learning plays a pivotal role in modeling evolving relationships over time, especially for temporal link prediction tasks in domains such as traffic systems, social networks, and recommendation platforms. While Transformer-based models have demonstrated strong performance by capturing long-range temporal dependencies, their reliance on self-attention results in quadratic complexity with respect to sequence length, limiting scalability on high-frequency or large-scale graphs. In this work, we revisit the necessity of self-attention in dynamic graph modeling. Inspired by recent findings that attribute the success of Transformers more to their architectural design than attention itself, we propose GLFormer, a novel attention-free Transformer-style framework for dynamic graphs. GLFormer introduces an adaptive token mixer that performs context-aware local aggregation based on interaction order and time intervals. To capture long-term dependencies, we further design a hierarchical aggregation module that expands the temporal receptive field by stacking local token mixers across layers. Experiments on six widely-used dynamic graph benchmarks show that GLFormer achieves SOTA performance, which reveals that attention-free architectures can match or surpass Transformer baselines in dynamic graph settings with significantly improved efficiency.

动态图Transformer注意力机制时序预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。