让自动驾驶模型像人一样一步步思考,提升复杂路况下的决策能力。
CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
- 引入显式思维链,让模型分步推理视觉、语言与动作关系。
- 在nuScenes和Bench2Drive上实现开环与闭环评估的领先性能。
- 适合需要可靠因果推理的自动驾驶系统研发人员参考。
视觉-语言-动作(VLA)模型因强大的推理能力和丰富的世界知识,近年来在端到端自动驾驶中备受关注。然而,现有VLA普遍存在数值推理能力不足和输入输出映射过于简化的问题,难以应对需要逐步因果推理的复杂驾驶场景。为此,我们提出CoT4AD,一种新型VLA框架,通过为自动驾驶引入思维链(CoT)推理,增强视觉语言模型在数值和因果推理方面的能力。CoT4AD融合视觉观测与语言指令,实现语义推理、场景理解与轨迹规划。训练时,显式建模感知-提问-预测-动作的思维链,使推理空间与动作空间对齐;推理时,执行隐式思维链推理,确保动态环境中的稳定数值推理与鲁棒决策。在真实世界与仿真基准(nuScenes、Bench2Drive)上的大量实验表明,CoT4AD在开环与闭环评估中均达到当前最优表现。代码将在论文接收后发布。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have recently attracted growing attention in end-to-end autonomous driving for their strong reasoning capabilities and rich world knowledge. However, existing VLAs often suffer from limited numerical reasoning ability and overly simplified input-output mappings, which hinder their performance in complex driving scenarios requiring step-by-step causal reasoning. To address these challenges, we propose CoT4AD, a novel VLA framework that introduces Chain-of-Thought (CoT) reasoning for autonomous driving to enhance both numerical and causal reasoning in Vision-Language Models (VLMs). CoT4AD integrates visual observations and language instructions to perform semantic reasoning, scene understanding, and trajectory planning. During training, it explicitly models a perception-question-prediction-action CoT to align the reasoning space with the action space across multiple driving tasks. During inference, it performs implicit CoT reasoning to enable consistent numerical reasoning and robust decision-making in dynamic environments. Extensive experiments on both real-world and simulated benchmarks, including nuScenes and Bench2Drive, demonstrate that CoT4AD achieves state-of-the-art performance in both open-loop and closed-loop evaluations. Code will be released upon paper acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。