用多模态大模型分析轨迹数据,不需训练就能跨区域通用。
Traj-MLLM: Can Multimodal Large Language Models Reform Trajectory Data Mining?
- 将轨迹转为图文交织序列,利用大模型推理能力
- 在4个数据集上性能超越基线48%~52%不等
- 无需微调,适合多种轨迹分析任务
构建能跨地理区域和任务通用的轨迹分析模型成为诸多应用中的关键问题。现有方法普遍存在泛化能力差的问题,或仅限于特定区域,或仅适用于少数任务。借助多模态大语言模型(MLLMs)的发展,我们提出:能否用MLLM重构轨迹数据挖掘?然而,轨迹与视觉/文本模态存在差异,如何生成任务无关的多模态表示并灵活适应不同任务仍是核心挑战。本文提出首个基于MLLM的通用轨迹挖掘框架Traj-MLLM。通过融合多视角上下文,该框架将原始轨迹转换为保留时空特征的图文交错序列,并直接利用MLLM的推理能力进行分析。同时,提出提示优化方法以生成数据无关的提示用于任务适配。在四个公开数据集上的实验表明,Traj-MLLM在旅行时间预测、移动性预测、异常检测和交通方式识别任务上分别优于当前最优基线48.05%、15.52%、51.52%、1.83%。该框架无需任何训练或微调大模型主干网络即可实现优异表现。
原文摘要 · Abstract (English)
Building a general model capable of analyzing human trajectories across different geographic regions and different tasks becomes an emergent yet important problem for various applications. However, existing works suffer from the generalization problem, \ie, they are either restricted to train for specific regions or only suitable for a few tasks. Given the recent advances of multimodal large language models (MLLMs), we raise the question: can MLLMs reform current trajectory data mining and solve the problem? Nevertheless, due to the modality gap of trajectory, how to generate task-independent multimodal trajectory representations and how to adapt flexibly to different tasks remain the foundational challenges. In this paper, we propose \texttt{Traj-MLLM}}, which is the first general framework using MLLMs for trajectory data mining. By integrating multiview contexts, \texttt{Traj-MLLM}} transforms raw trajectories into interleaved image-text sequences while preserving key spatial-temporal characteristics, and directly utilizes the reasoning ability of MLLMs for trajectory analysis. Additionally, a prompt optimization method is proposed to finalize data-invariant prompts for task adaptation. Extensive experiments on four publicly available datasets show that \texttt{Traj-MLLM}} outperforms state-of-the-art baselines by $48.05\%$, $15.52\%$, $51.52\%$, $1.83\%$ on travel time estimation, mobility prediction, anomaly detection and transportation mode identification, respectively. \texttt{Traj-MLLM}} achieves these superior performances without requiring any training data or fine-tuning the MLLM backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。