通过收集人类接管数据并优化决策，提升自动驾驶在罕见危险场景下的表现。
CoReVLA: A Dual-Stage End-to-End Autonomous Driving Framework for Long-Tail Scenarios via Collect-and-Refine
- 分两阶段：先用公开数据集打基础，再在仿真中收集失败案例。
- 在Bench2Drive上达成72.18分驾驶得分和50%成功率，优于当前最佳方法。
- 直接基于人类偏好优化，避免奖励欺骗，适合长期迭代的自动驾驶系统。
自动驾驶系统虽有显著进展，但在长尾、高风险场景下的表现仍受限，而这些罕见情况恰恰导致了大量事故。视觉-语言-动作（VLA）模型具备强大推理能力，但受限于高质量数据不足与学习效率低下。为此，我们提出CoReVLA，一种持续学习的端到端自动驾驶框架，通过数据收集与行为精炼双阶段提升长尾场景性能。首先，在多个开源驾驶问答数据集上联合微调，建立基础驾驶理解；其次，在CAVE仿真平台部署，实时收集驾驶员接管数据，每条接管对应一次模型未能可靠处理的长尾场景；最后，采用直接偏好优化（DPO）对模型进行精炼，使其直接学习人类偏好，避免人工设计奖励带来的奖励黑客问题。大量开环与闭环实验表明，CoReVLA能准确感知驾驶场景并做出合理决策。在Bench2Drive基准上，其驾驶得分（DS）达72.18，成功率达50%，相比当前最优方法提升7.96分和15个百分点。案例研究进一步证明，模型可通过过往接管经验持续改进类似故障场景的应对能力。所有代码与预处理数据集见：https://github.com/FanGShiYuu/CoReVLA
原文摘要 · Abstract (English)
Autonomous Driving (AD) systems have made notable progress, but their performance in long-tail, safety-critical scenarios remains limited. These rare cases contribute a disproportionate number of accidents. Vision-Language Action (VLA) models have strong reasoning abilities and offer a potential solution, but their effectiveness is limited by the lack of high-quality data and inefficient learning in such conditions. To address these challenges, we propose CoReVLA, a continual learning end-to-end autonomous driving framework that improves the performance in long-tail scenarios through a dual-stage process of data Collection and behavior Refinement. First, the model is jointly fine-tuned on a mixture of open-source driving QA datasets, allowing it to acquire a foundational understanding of driving scenarios. Next, CoReVLA is deployed within the Cave Automatic Virtual Environment (CAVE) simulation platform, where driver takeover data is collected from real-time interactions. Each takeover indicates a long-tail scenario that CoReVLA fails to handle reliably. Finally, the model is refined via Direct Preference Optimization (DPO), allowing it to learn directly from human preferences and thereby avoid reward hacking caused by manually designed rewards. Extensive open-loop and closed-loop experiments demonstrate that the proposed CoReVLA model can accurately perceive driving scenarios and make appropriate decisions. On the Bench2Drive benchmark, CoReVLA achieves a Driving Score (DS) of 72.18 and a Success Rate (SR) of 50%, outperforming state-of-the-art methods by 7.96 DS and 15% SR under long-tail, safety-critical scenarios. Furthermore, case studies demonstrate the model's ability to continually improve its performance in similar failure-prone scenarios by leveraging past takeover experiences. All codea and preprocessed datasets are available at: https://github.com/FanGShiYuu/CoReVLA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。