让自动驾驶模型根据自然语言指令预测6秒轨迹，提升指令与视觉的对齐精度。
A DVDrive Approach for doScenes Instructed Driving Challenge
- 基于多视角分块感知，强化语言指令与局部视觉证据的对齐
- 在nuScenes数据集上生成12个未来位点的6秒轨迹，精度显著提升
- 适合关注视觉-语言融合与自动驾驶决策的开发者和研究者
指令条件下的轨迹预测是自动驾驶中的新兴问题，要求模型不仅依据视觉场景和历史运动，还需结合自然语言指令预测未来的车辆轨迹。本文提交了对doScenes指令驱动挑战赛的解决方案，基于OmniDrive——一个具备3D感知、推理与规划能力的视觉-语言-动作驾驶代理。我们通过在带指令标注的nuScenes场景上训练，使模型生成由12个未来位点组成的6秒自车轨迹。为进一步提升多视角视觉定位能力，我们在OmniDrive的感知头中引入类似DVPE的分视图感知模块：不再全局关注所有相机特征，而是将查询特征与图像标记分组至局部视图空间，并在每个视图内进行可见性感知的跨注意力计算。该设计减少了跨视图无关干扰，帮助模型更精准地将语言指令与相关视觉线索对齐。代码已公开于：https://github.com/feel12348/doscenes-omnidrive。
原文摘要 · Abstract (English)
Instruction-conditioned trajectory prediction is an emerging problem in autonomous driving, where a model predicts the future ego trajectory not only from visual scene context and historical motion, but also from a natural-language maneuver instruction. This paper presents our submission to the doScenes Instructed Driving Challenge, built upon OmniDrive, a vision-language-action driving agent with 3D perception, reasoning, and planning capabilities. We adapt OmniDrive to the doScenes setting by training it on instruction-annotated nuScenes scenes and generating a 6-second ego trajectory represented by 12 future waypoints. To improve multi-view visual grounding, we further introduce a DVPE-style divided-view perception module into the OmniDrive perception head. Instead of attending globally to all camera features, the proposed module groups query features and image tokens into divided local view spaces and performs visibility-aware cross-attention within each view. This design reduces irrelevant cross-view interference and helps the model better align language instructions with local driving-relevant visual evidence. The code is publicly available at: https://github.com/feel12348/doscenes-omnidrive.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。