用大模型推理人类行为，实现更准的机器人轨迹预测。
AutoTraces: Autoregressive Trajectory Forecasting via Multimodal Large Language Models
- 将轨迹点拆分为类别与位置标记，融入大模型生成机制。
- 长时序预测准确率达最优，跨场景泛化能力强。
- 自动推理解析视觉与轨迹关系，无需人工标注。
我们提出 AutoTraces，一种基于多模态大语言模型的自回归视觉-语言-轨迹模型，用于人机共存环境中的机器人轨迹预测。不同于仅依赖文本表示的现有方法，其核心创新在于一种新型轨迹分词方案：将路径点以类别标记和位置信息表示，数值坐标则通过轻量编码器-解码器映射为点嵌入，无缝融入大模型空间。该设计保留大模型原有的自回归生成机制，同时扩展至物理坐标空间，有效建模轨迹数据中的长期交互关系。我们进一步引入自动化思维链（CoT）生成机制，利用多模态大模型从视觉观测与轨迹数据中推断时空关联，避免依赖人工标注。通过两阶段训练策略，AutoTraces 在长时程预测上达到当前最优性能，展现出强跨场景泛化能力，并支持灵活长度的轨迹生成。
原文摘要 · Abstract (English)
We present AutoTraces, an autoregressive vision-language-trajectory model for robot trajectory forecasting in humam-populated environments, which harnesses the inherent reasoning capabilities of large language models (LLMs) to model complex human behaviors. In contrast to prior works that rely solely on textual representations, our key innovation lies in a novel trajectory tokenization scheme, which represents waypoints with point tokens as categorical and positional markers while encoding waypoint numerical values as corresponding point embeddings, seamlessly integrated into the LLM's space through a lightweight encoder-decoder architecture. This design preserves the LLM's native autoregressive generation mechanism while extending it to physical coordinate spaces, facilitates modeling of long-term interactions in trajectory data. We further introduce an automated chain-of-thought (CoT) generation mechanism that leverages a multimodal LLM to infer spatio-temporal relationships from visual observations and trajectory data, eliminating reliance on manual annotation. Through a two-stage training strategy, our AutoTraces achieves SOTA forecasting accuracy, particularly in long-horizon prediction, while exhibiting strong cross-scene generalization and supporting flexible-length forecasting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。