用分层时间分词让大模型更高效预测人类移动轨迹
RHYTHM: Reasoning with Hierarchical Temporal Tokenization for Human Mobility
- 将轨迹按天分割成离散令牌，用层次注意力捕捉日周周期依赖
- 准确率提升2.4%，周末预测提升5.0%，训练时间减少24.6%
- 适合需要高效处理长序列移动数据的研究与应用
由于复杂的长程依赖和多尺度周期行为，预测人类移动极具挑战。我们提出RHYTHM（基于分层时间分词的人类移动推理），一个统一框架，利用大型语言模型（LLMs）作为通用时空预测器和轨迹推理器。方法上，RHYTHM采用时间分词，将每条轨迹划分为每日片段，并以分层注意力编码为离散令牌，从而将序列长度降低四倍，同时保留循环信息。此外，通过冻结的LLM预计算轨迹片段和预测目标的提示嵌入，增强令牌表示，并将其反馈至LLM主干网络，以捕获复杂依赖关系。计算上，保持预训练的LLM主干冻结，实现更快训练与更低内存消耗。我们在三个真实世界数据集上评估模型，结果表明RHYTHM在整体准确率上提升2.4%，周末表现提高5.0%，训练时间减少24.6%。代码已公开于https://github.com/he-h/rhythm。
原文摘要 · Abstract (English)
Predicting human mobility is inherently challenging due to complex long-range dependencies and multi-scale periodic behaviors. To address this, we introduce RHYTHM (Reasoning with Hierarchical Temporal Tokenization for Human Mobility), a unified framework that leverages large language models (LLMs) as general-purpose spatio-temporal predictors and trajectory reasoners. Methodologically, RHYTHM employs temporal tokenization to partition each trajectory into daily segments and encode them as discrete tokens with hierarchical attention that captures both daily and weekly dependencies, thereby quadratically reducing the sequence length while preserving cyclical information. Additionally, we enrich token representations by adding pre-computed prompt embeddings for trajectory segments and prediction targets via a frozen LLM, and feeding these combined embeddings back into the LLM backbone to capture complex interdependencies. Computationally, RHYTHM keeps the pretrained LLM backbone frozen, yielding faster training and lower memory usage. We evaluate our model against state-of-the-art methods using three real-world datasets. Notably, RHYTHM achieves a 2.4% improvement in overall accuracy, a 5.0% increase on weekends, and a 24.6% reduction in training time. Code is publicly available at https://github.com/he-h/rhythm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。