让视觉语言与轨迹预测紧密协同,提升端到端自动驾驶性能
VECTOR-Drive: Tightly Coupled Vision-Language and Trajectory Expert Routing for End-to-End Autonomous Driving

- 用语义感知专家路由,分治视觉语言与运动计算
- 在Bench2Drive上达88.91分,超越主流端到端与VLA模型
- 适合研究多模态融合与自动驾驶规划的开发者
端到端自动驾驶需理解交通场景、推断驾驶意图并生成可执行运动规划。现有视觉-语言-动作(VLA)模型虽继承大规模视觉-语言预训练的语义先验,但仍面临耦合权衡:完全共享主干保留多模态交互但可能混淆语言推理与轨迹预测,而解耦的推理-动作流程虽减少任务冲突却削弱语义-运动关联。本文提出VECTOR-DRIVE,基于Qwen2.5-VL-3B构建紧密耦合的VLA框架。所有令牌通过共享自注意力保持耦合,并根据令牌语义路由前馈计算。视觉与语言令牌由视觉-语言专家处理以保留语义先验,目标点、自身状态及噪声动作令牌则路由至轨迹专家进行运动特异性计算。在动作令牌路径上,流匹配规划器将噪声动作令牌精炼为未来航点与速度曲线。该设计在单一多模态Transformer内实现语义推理与运动规划的紧密耦合,同时分离任务特异性前馈网络计算。在Bench2Drive数据集上,VECTOR-DRIVE取得88.91分驾驶得分,优于代表性端到端与VLA基线。定性结果与消融实验进一步验证了共享注意力、语义感知专家路由、渐进式训练与基于流的动作解码的优势。
原文摘要 · Abstract (English)
End-to-end autonomous driving requires models to understand traffic scenes, infer driving intent, and generate executable motion plans. Recent vision-language-action (VLA) models inherit semantic priors from large-scale vision-language pretraining, yet still face a coupling trade-off: fully shared backbones preserve multimodal interaction but may entangle language reasoning and trajectory prediction, whereas decou pled reasoning-action pipelines reduce task conflict but weaken semantic-motion coupling. We propose VECTOR-DRIVE, a tightly coupled VLA framework built on Qwen2.5-VL-3B. VECTOR-DRIVE keeps all tokens coupled through shared self attention and routes feed-forward computation according to token semantics. Vision and language tokens are processed by a Vision-Language Expert to preserve semantic priors, while target-point, ego-state, and noisy action tokens are routed to a Trajectory Expert for motion-specific computation. On the action-token pathway, a flow-matching planner refines noisy action tokens into future waypoints and speed profiles. This design couples semantic reasoning and motion planning within a single multimodal Transformer while separating task-specific FFN computation. On Bench2Drive, VECTOR-DRIVE achieves 88.91 Driving Score and outperforms representative end-to end and VLA-based baselines. Qualitative results and ablations further validate the benefits of shared attention, semantic-aware expert routing, progressive training, and flow-based action de coding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。