arXiv:2509.13753cs.LG2025-09中稿 · CIKM 2025被引 5

让大模型学会看地图,精准预测交通流量变化。

ST-LINK: Spatially-Aware Large Language Models for Spatio-Temporal Forecasting

  • 用空间注意力机制增强大模型对地理关系的感知能力。
  • 在多个基准数据集上超越传统深度学习和大模型方法。
  • 适合需要处理时空数据的智能交通系统研究者使用。

交通预测是智能交通系统中的关键问题。近年来,大语言模型(LLM)展现出广阔前景,但其固有的序列化令牌处理设计难以有效捕捉空间依赖性。具体表现为:LLM在建模空间关系方面存在内在局限,且与图结构空间数据的架构不兼容问题尚未解决。为此,我们提出ST-LINK框架,提升大模型对时空依赖性的建模能力。核心组件包括空间增强注意力(SE-Attention)和记忆检索前馈网络(MRFFN)。SE-Attention将旋转位置编码扩展为直接的空间相关性旋转变换,嵌入注意力机制中,在保持原有序列处理结构的同时最大化空间学习。MRFFN则动态检索并利用关键历史模式,以捕捉复杂时间依赖性并提升长期预测稳定性。在多个基准数据集上的综合实验表明,ST-LINK优于传统深度学习与大模型方法,能有效捕捉规律性交通模式及突发变化。

原文摘要 · Abstract (English)

Traffic forecasting represents a crucial problem within intelligent transportation systems. In recent research, Large Language Models (LLMs) have emerged as a promising method, but their intrinsic design, tailored primarily for sequential token processing, introduces notable challenges in effectively capturing spatial dependencies. Specifically, the inherent limitations of LLMs in modeling spatial relationships and their architectural incompatibility with graph-structured spatial data remain largely unaddressed. To overcome these limitations, we introduce ST-LINK, a novel framework that enhances the capability of Large Language Models to capture spatio-temporal dependencies. Its key components are Spatially-Enhanced Attention (SE-Attention) and the Memory Retrieval Feed-Forward Network (MRFFN). SE-Attention extends rotary position embeddings to integrate spatial correlations as direct rotational transformations within the attention mechanism. This approach maximizes spatial learning while preserving the LLM's inherent sequential processing structure. Meanwhile, MRFFN dynamically retrieves and utilizes key historical patterns to capture complex temporal dependencies and improve the stability of long-term forecasting. Comprehensive experiments on benchmark datasets demonstrate that ST-LINK surpasses conventional deep learning and LLM approaches, and effectively captures both regular traffic patterns and abrupt changes.

交通预测大模型时空建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。