用分层时间分块让大模型更高效预测人类移动轨迹
Efficient Temporal Tokenization for Mobility Prediction with Large Language Models
- 将轨迹按天分割成离散令牌，用层次注意力捕捉日周依赖
- 在三个真实数据集上准确率提升2.4%，周末效果提高5.0%
- 冻结大模型主干，训练时间减少24.6%，适合资源受限场景
我们提出RHYTHM（基于分层时间分块的人类移动推理框架），利用大语言模型（LLMs）作为时空预测器与轨迹推理器。RHYTHM将轨迹划分为每日片段，并以离散令牌编码，通过层次注意力机制同时捕捉日级与周级依赖关系，显著降低序列长度。令牌表示通过冻结的LLM预计算提示嵌入进行增强，提升模型对跨依赖关系的建模能力，且无额外计算开销。由于冻结了LLM主干，RHYTHM实现显著的计算效率。在三个真实世界数据集上的评估表明，相比现有最优方法，准确率提升2.4%，周末表现提高5.0%，训练时间减少24.6%。
原文摘要 · Abstract (English)
We introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a framework that leverages large language models (LLMs) as spatio-temporal predictors and trajectory reasoners. RHYTHM partitions trajectories into daily segments encoded as discrete tokens with hierarchical attention, capturing both daily and weekly dependencies while substantially reducing the sequence length. Token representations are enriched with pre-computed prompt embeddings via a frozen LLM, enhancing the model's ability to capture interdependencies without extensive computational overhead. By freezing the LLM backbone, RHYTHM achieves significant computational efficiency. Evaluation on three real-world datasets demonstrates a 2.4% improvement in accuracy, 5.0% increase on weekends, and 24.6% reduction in training time compared to state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。