让自动驾驶模型学会该思考时思考,不该时就不想,又快又准。
AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Driving
- 用快慢思维双模式训练模型,自动判断何时需要推理。
- 在Navsim上达90.3的PDMS,比纯视觉基线高1.7点。
- 推理时间降14%,适合追求效率与准确平衡的落地场景。
尽管链式思维(CoT)在视觉语言动作(VLA)模型中广泛应用,但在端到端自动驾驶中表现参差不齐:简单场景下引入冗余计算却未提升决策质量。为此,我们提出AdaThinkDrive,一种受快慢思维启发的双模式VLA框架。首先,在大规模自动驾驶场景下联合使用问答与轨迹数据预训练,获取世界知识与驾驶常识。在监督微调阶段,引入快答(无CoT)与慢思(含CoT)双模式数据集,使模型学会识别需推理的场景。同时结合组相对策略优化(GRPO),设计自适应思考奖励机制,通过对比不同推理模式下的轨迹质量,激励模型选择性启用CoT。Navsim基准测试显示,AdaThinkDrive达90.3的PDMS,优于最佳纯视觉基线1.7点。消融实验表明,其性能分别超越从不思考与始终思考基线2.0与1.4点,且推理时间减少14%,证明了其在精度与效率间的良好权衡能力。
原文摘要 · Abstract (English)
While reasoning technology like Chain of Thought (CoT) has been widely adopted in Vision Language Action (VLA) models, it demonstrates promising capabilities in end to end autonomous driving. However, recent efforts to integrate CoT reasoning often fall short in simple scenarios, introducing unnecessary computational overhead without improving decision quality. To address this, we propose AdaThinkDrive, a novel VLA framework with a dual mode reasoning mechanism inspired by fast and slow thinking. First, our framework is pretrained on large scale autonomous driving (AD) scenarios using both question answering (QA) and trajectory datasets to acquire world knowledge and driving commonsense. During supervised fine tuning (SFT), we introduce a two mode dataset, fast answering (w/o CoT) and slow thinking (with CoT), enabling the model to distinguish between scenarios that require reasoning. Furthermore, an Adaptive Think Reward strategy is proposed in conjunction with the Group Relative Policy Optimization (GRPO), which rewards the model for selectively applying CoT by comparing trajectory quality across different reasoning modes. Extensive experiments on the Navsim benchmark show that AdaThinkDrive achieves a PDMS of 90.3, surpassing the best vision only baseline by 1.7 points. Moreover, ablations show that AdaThinkDrive surpasses both the never Think and always Think baselines, improving PDMS by 2.0 and 1.4, respectively. It also reduces inference time by 14% compared to the always Think baseline, demonstrating its ability to balance accuracy and efficiency through adaptive reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。