让自动驾驶模型像多专家协作一样思考,提升规划准确性与安全性。
CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving

- 用四个专家令牌分别捕捉交互、结构、动态和轨迹信息,实现可规划的环境表征
- 在NAVSIM v1/v2上实现零碰撞与高轨迹精度,优于传统文本推理方法
- 适合研究端到端自动驾驶规划、多智能体协同决策的开发者
视觉-语言-动作(VLA)模型已成为端到端自动驾驶的有前景范式。然而,现有推理机制难以提供面向规划的中间表示:文本链式思维(CoT)无法保留连续时空结构,潜空间世界推理也难直接用于动作生成。本文提出CoWorld-VLA,一种多专家世界推理框架,通过显式世界表征作为规划条件。该模型通过多源监督提取互补世界信息,并编码为VLA中的专家令牌,包括语义交互、几何结构、动态演化和自车轨迹四类令牌,分别建模交互意图、空间结构、未来时序动态与行为目标。在动作生成阶段,采用基于扩散的分层多专家融合规划器,结合场景上下文进行联合去噪,生成连续自车轨迹。在NAVSIM v1与v2上的实验验证了其未来场景建模能力及强规划性能,包括零碰撞与高轨迹精度。消融实验进一步证实各专家令牌的互补性及其作为规划条件的有效性。代码将发布于https://github.com/AFARI-Research/CoWorld-VLA。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have emerged as a promising paradigm for end-to-end autonomous driving. However, existing reasoning mechanisms still struggle to provide planning-oriented intermediate representations: textual Chain-of-Thought (CoT) fails to preserve continuous spatiotemporal structure, while latent world reasoning remains difficult to use as a direct condition for action generation. In this paper, we propose CoWorld-VLA, a multi-expert world reasoning framework for autonomous driving, where world representations serve as explicit conditions to guide action planning. CoWorld-VLA extracts complementary world information through multi-source supervision and encodes it into expert tokens within the VLA, thereby providing planner-accessible conditioning signals. Specifically, we construct four types of tokens: semantic interaction, geometric structure, dynamic evolution, and ego trajectory tokens, which respectively model interaction intent, spatial structure, future temporal dynamics, and behavioral goals. During action generation, CoWorld-VLA employs a diffusion-based hierarchical multi-expert fusion planner, which is coupled with scene context throughout the joint denoising process to generate continuous ego trajectories. Experiments on NAVSIM v1 & v2 demonstrate future-scene modeling capability and strong planning performance, including collision avoidance and trajectory accuracy. Ablation studies further validate the complementarity of expert tokens and their effectiveness as planning conditions for action generation. Code will be available at https://github.com/AFARI-Research/CoWorld-VLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。