用视觉语言模型融合交通图块数据,提升城市交通预测精度。
Vision-LLMs for Spatiotemporal Traffic Forecasting
- 将交通矩阵当图像序列处理,通过视觉编码器捕捉全局空间关系。
- 用单标记编码浮点数,使大模型高效处理数值数据,长期预测准确率高15.6%。
- 适合数据少的场景,跨域少量样本下表现优于现有方法约30%。
精准的时空交通预测是密集城市移动网络中主动资源管理的关键前提。尽管大语言模型在时间序列分析中展现潜力,但其难以建模基于网格的交通数据中的复杂空间依赖。将大语言模型有效扩展到该领域面临挑战:稠密地理网格产生的海量信息表示效率低,易超出模型上下文容量。为此,我们提出ST-Vision-LLM,一种将时空预测重构为视觉-语言融合问题的新框架。该方法利用视觉-语言模型的视觉编码器处理历史全局交通矩阵作为图像序列,为单元级预测提供全面全局视图。为克服大语言模型处理数值数据的效率问题,我们引入一种高效编码方案,通过专用词汇表将浮点值表示为单个标记,并结合两阶段数值对齐微调流程。模型首先经监督微调训练,再通过组相对策略优化(group relative policy optimization)进一步优化预测准确性,这是一种内存高效的强化学习方法。在真实移动交通数据集上的评估显示,ST-Vision-LLM在长期预测中比现有方法准确率提升15.6%,在跨域少样本场景下平均性能超越最佳基线约30%。大量实验验证了模型在多种数据稀缺环境下的强泛化能力。
原文摘要 · Abstract (English)
Accurate spatiotemporal traffic forecasting is a critical prerequisite for proactive resource management in dense urban mobile networks. While large language models have shown promise in time series analysis, they inherently struggle to model the complex spatial dependencies of grid-based traffic data. Effectively extending large language models to this domain is challenging, as representing the vast amount of information from dense geographical grids can be inefficient and overwhelm the model's context. To address these challenges, we propose ST-Vision-LLM, a novel framework that reframes spatiotemporal forecasting as a vision-language fusion problem. Our approach leverages a Vision-LLM visual encoder to process historical global traffic matrices as image sequences, providing the model with a comprehensive global view to inform cell-level predictions. To overcome the inefficiency of large language models in handling numerical data, we introduce an efficient encoding scheme that represents floating-point values as single tokens via a specialized vocabulary, coupled with a two-stage numerical alignment fine-tuning process. The model is first trained with supervised fine-tuning and then further optimized for predictive accuracy using group relative policy optimization, a memory-efficient reinforcement learning method. Evaluations on real-world mobile traffic datasets demonstrate that ST-Vision-LLM outperforms existing methods by 15.6% in long-term prediction accuracy and exceeds the best baseline by around 30% on average in cross-domain few-shot scenarios. Our extensive experiments validate the model's strong generalization capabilities across various data-scarce environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。