arXiv:2603.06254cs.CVcs.RO2026-03中稿 · IROS 2026

用语言模型让自动驾驶跟踪未知物体,准确率提升20%。

NOVA: Next-step Open-Vocabulary Autoregression for 3D Multi-Object Tracking in Autonomous Driving

  • 将轨迹建模为时空语义序列,用语言推理解决跟踪难题
  • 在nuScenes上对新类别物体追踪准确率提升20.21个百分点
  • 仅用0.5亿参数的小模型实现开放词汇跟踪

泛化到未知目标是开放世界感知的关键,但现有3D多目标跟踪(3D MOT)方法仍受限于封闭集假设和“语义盲”启发式规则。为此,我们提出下一阶段开放词汇自回归(NOVA),一种将数据关联从碎片化的距离匹配转向轨迹条件的时空语义建模的自回归形式。NOVA将3D轨迹重构为结构化的时空语义序列,同时编码物理运动连续性和深层语言先验。通过利用大语言模型(LLMs)的自回归能力,我们将追踪任务转化为下一阶段序列补全的严谨过程。该机制使模型能显式利用语言空间的层次结构,解决细粒度语义模糊性,并通过高层常识推理在复杂长序列中保持身份一致性。在nuScenes、V2X-Seq-SPD和KITTI上的大量实验表明,NOVA性能优越。尤其在nuScenes数据集上,对新类别物体的AMOTA达到22.41%,相比基线提升20.21%绝对值。这些成果由一个仅0.5B参数的紧凑自回归模型实现。代码将发布于https://github.com/xifen523/NOVA。

原文摘要 · Abstract (English)

Generalizing across unknown targets is critical for open-world perception, yet existing 3D Multi-Object Tracking (3D MOT) pipelines remain limited by closed-set assumptions and ``semantic-blind'' heuristics. To address this, we propose Next-step Open-Vocabulary Autoregression (NOVA), an autoregressive association formulation that shifts the data association stage from fragmented distance-based matching toward trajectory-conditioned spatio-semantic modeling. NOVA reformulates 3D trajectories as structured spatio-temporal semantic sequences, enabling the simultaneous encoding of physical motion continuity and deep linguistic priors. By leveraging the autoregressive capabilities of Large Language Models (LLMs), we transform the tracking task into a principled process of next-step sequence completion. This mechanism allows the model to explicitly utilize the hierarchical structure of language space to resolve fine-grained semantic ambiguities and maintain identity consistency across complex long-range sequences through high-level commonsense reasoning. Extensive experiments on nuScenes, V2X-Seq-SPD, and KITTI demonstrate the superior performance of NOVA. Notably, on the nuScenes dataset, NOVA achieves an AMOTA of 22.41% for Novel categories, yielding a significant 20.21% absolute improvement over the baseline. These gains are realized through a compact 0.5B autoregressive model. Code will be available at https://github.com/xifen523/NOVA.

3D跟踪开放词汇自回归自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。