用语言推理+在线强化学习,让自动驾驶模型边试边学更安全。
MindDrive: A Vision-Language-Action Model for Autonomous Driving via Online Reinforcement Learning
- 用大模型分角色:一个管决策,一个管动作,降低探索难度。
- 在Bench2Drive上达78.04分驾驶得分,55.09%成功率,性能领先。
- 首次实现自动驾驶VLA模型的在线强化学习,适合研究高效智能驾驶者。
当前自动驾驶中的视觉-语言-动作(VLA)范式主要依赖模仿学习,存在分布偏移和因果混淆等固有挑战。在线强化学习通过试错机制可缓解这些问题,但其在连续动作空间中探索效率低。为此,我们提出MindDrive,一种基于大语言模型(LLM)的VLA框架,配备两组独立的LoRA参数。其中一组作为决策专家,负责场景理解与驾驶决策;另一组作为动作专家,将语言决策动态映射为可行轨迹。通过将轨迹级奖励反馈至推理空间,MindDrive在有限离散语言决策集上实现试错学习,而非直接在连续动作空间操作。该方法有效平衡了复杂场景下的最优决策、类人驾驶行为与高效探索。采用轻量级Qwen-0.5B LLM,MindDrive在挑战性Bench2Drive基准上取得78.04分驾驶得分(DS)与55.09%成功率(SR)。据我们所知,这是首个证明在线强化学习在自动驾驶VLA模型中有效的工作。
原文摘要 · Abstract (English)
Current Vision-Language-Action (VLA) paradigms in autonomous driving primarily rely on Imitation Learning (IL), which introduces inherent challenges such as distribution shift and causal confusion. Online Reinforcement Learning offers a promising pathway to address these issues through trial-and-error learning. However, applying online reinforcement learning to VLA models in autonomous driving is hindered by inefficient exploration in continuous action spaces. To overcome this limitation, we propose MindDrive, a VLA framework comprising a large language model (LLM) with two distinct sets of LoRA parameters. The one LLM serves as a Decision Expert for scenario reasoning and driving decision-making, while the other acts as an Action Expert that dynamically maps linguistic decisions into feasible trajectories. By feeding trajectory-level rewards back into the reasoning space, MindDrive enables trial-and-error learning over a finite set of discrete linguistic driving decisions, instead of operating directly in a continuous action space. This approach effectively balances optimal decision-making in complex scenarios, human-like driving behavior, and efficient exploration in online reinforcement learning. Using the lightweight Qwen-0.5B LLM, MindDrive achieves Driving Score (DS) of 78.04 and Success Rate (SR) of 55.09% on the challenging Bench2Drive benchmark. To the best of our knowledge, this is the first work to demonstrate the effectiveness of online reinforcement learning for the VLA model in autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。