arXiv:2607.04812cs.CV2026-07

用文本提示增强车辆实例预测,让自动驾驶更懂复杂场景行为。

TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving

论文配图:TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving
图 1 · 摘自论文原文
  • 用视觉语言模型生成带语义的鸟瞰图地图,作为训练辅助监督。
  • 在nuScenes数据集上超越现有最佳模型,提升复杂场景预测准确率。
  • 适合研究端到端自动驾驶感知与语义引导模型的学者和工程师。

鸟瞰图(BEV)端到端实例预测已成为自动驾驶感知的稳健范式,有效缓解了传统模块化流程中的误差传播问题。然而,当前最先进的方法主要依赖几何监督(如占据回归和光流),将场景主体视为通用移动障碍物,缺乏显式语义意识,限制了模型在复杂场景中解决歧义的能力,尤其在需要特定物体行为理解的场景(如变道、交叉路口)中表现不足。本文提出文本引导实例预测框架TGRIP,通过注入丰富的语义先验信息,弥补这一差距。该框架采用师生协同架构,利用视觉语言基础模型从多摄像头图像生成密集语义增强的鸟瞰图地图,并将其作为训练阶段的辅助监督信号,引导网络学习不仅几何一致且语义区分性强的时空表示。据我们所知,这是首次将语义引导与未来实例预测任务统一的尝试。实验结果表明,TGRIP在nuScenes数据集上优于现有最先进模型,验证了语义丰富性是鲁棒端到端运动预测的关键要素。代码已开源:https://github.com/miguelag99/TGRIP。

原文摘要 · Abstract (English)

Bird's-Eye View (BEV) end-to-end instance prediction has emerged as a robust paradigm for autonomous driving perception, effectively mitigating the error propagation inherent in traditional modular pipelines. However, current state-of-the-art approaches rely predominantly on geometric supervision, such as occupancy regression and optical flow, effectively treating scene agents as generic moving obstacles. This absence of explicit semantic awareness imposes limitations on the capacity of the model to solve ambiguities in complex scenarios, particularly those where object-specific behavior is essential for accurate forecasting (e.g. overtaking, intersections). In this paper, we introduce Text-Guided Representation for Instance Prediction (TGRIP), a novel framework that bridges this gap by injecting rich semantic priors into the instance prediction loop. The proposed teacher-student pipeline employs Vision-Language Foundation Models to generate dense, semantic-enhanced BEV maps from multi-camera images. These maps serve as auxiliary supervision during training, guiding the network to learn spatio-temporal representations that are not only geometrically consistent but also semantically discriminative. To the best of our knowledge, this represents the first attempt to unify semantic guidance with the temporal task of future instance prediction. The experimental results demonstrate that TGRIP surpasses existing state-of-the-art models in nuScenes, validating the hypothesis that semantic enrichment is a fundamental element for robust, end-to-end motion prediction. Code is available on https://github.com/miguelag99/TGRIP.

自动驾驶实例预测语义引导BEV

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。