用视觉语言模型生成自动驾驶轨迹,提速降延迟。
ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
- 将文本推理转为统一隐空间,仅两步前向传播完成场景理解
- 单次前向生成多尺度、满足因果性的并行轨迹,延迟显著降低
- 适合实时自动驾驶系统,兼顾安全与可解释性
自动驾驶需从复杂多模态输入中生成安全可靠的轨迹。传统模块化流程分离感知、预测与规划,而近期端到端系统虽联合学习三者,但基于视觉-语言模型(VLM)的规划器仍面临三大挑战:(i) 离散文本推理与连续控制间的不匹配;(ii) 自回归思维链解码带来的高延迟;(iii) 效率低或非因果的规划器限制实时部署。本文提出ColaVLA,一个统一的视觉-语言-动作框架,将推理从文本迁移至统一隐空间,并耦合分层并行轨迹解码器。认知隐式推理器通过自适应选择,仅用两次VLM前向传播,将场景理解压缩为紧凑的决策导向元动作嵌入。分层并行规划器则在单次前向传播中生成多尺度、满足因果一致性的轨迹。二者结合在保持VLM泛化性与可解释性的基础上,实现高效、准确且安全的轨迹生成。在nuScenes基准上的实验表明,ColaVLA在开环与闭环设置下均达到当前最优性能,兼具良好效率与鲁棒性。
原文摘要 · Abstract (English)
Autonomous driving requires generating safe and reliable trajectories from complex multimodal inputs. Traditional modular pipelines separate perception, prediction, and planning, while recent end-to-end (E2E) systems learn them jointly. Vision-language models (VLMs) further enrich this paradigm by introducing cross-modal priors and commonsense reasoning, yet current VLM-based planners face three key challenges: (i) a mismatch between discrete text reasoning and continuous control, (ii) high latency from autoregressive chain-of-thought decoding, and (iii) inefficient or non-causal planners that limit real-time deployment. We propose ColaVLA, a unified vision-language-action framework that transfers reasoning from text to a unified latent space and couples it with a hierarchical, parallel trajectory decoder. The Cognitive Latent Reasoner compresses scene understanding into compact, decision-oriented meta-action embeddings through ego-adaptive selection and only two VLM forward passes. The Hierarchical Parallel Planner then generates multi-scale, causality-consistent trajectories in a single forward pass. Together, these components preserve the generalization and interpretability of VLMs while enabling efficient, accurate and safe trajectory generation. Experiments on the nuScenes benchmark show that ColaVLA achieves state-of-the-art performance in both open-loop and closed-loop settings with favorable efficiency and robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。