提出Curious-VLA框架，解决自动驾驶VLA模型探索不足问题。
Devil is in Narrow Policy: Unleashing Exploration in Driving VLA Models
- 两阶段设计：模仿学习阶段扩增可行轨迹，强化学习阶段动态采样高多样性数据。
- 在Navsim上达90.3的PDMS和85.4的EPDMS，Best-of-N达94.8。
- 适合关注自动驾驶策略探索与强化学习效率的研究者。
我们发现自主视觉-语言-动作（VLA）模型性能受限于窄策略问题：驾驶模仿学习（IL）易导致探索坍缩，使后续强化学习（RL）因反馈多样性不足而过早饱和。为此，我们提出Curious-VLA框架，通过两阶段设计缓解探索-利用困境。在模仿学习阶段，引入可行轨迹扩展（FTE）生成多个物理可行轨迹，并采用分步归一化表示适配多样化数据；在强化学习阶段，提出自适应多样性感知采样（ADAS）优先选择高多样性样本，设计具有焦点加权的跨度驱动奖励（SDR），增强奖励值范围以提升对驾驶质量的敏感性。在Navsim基准上，Curious-VLA取得当前最优结果（PDMS 90.3，EPDMS 85.4），Best-of-N PDMS达94.8，充分验证其释放VLA模型探索潜力的有效性。代码已开源。
原文摘要 · Abstract (English)
We identify a fundamental Narrow Policy limitation undermining the performance of autonomous VLA models, where driving Imitation Learning (IL) tends to collapse exploration and limit the potential of subsequent Reinforcement Learning (RL) stages, which often saturate prematurely due to insufficient feedback diversity. Thereby, we propose Curious-VLA, a framework that alleviates the exploit-explore dilemma through a two-stage design. During IL, we introduce a Feasible Trajectory Expansion (FTE) strategy to generate multiple physically valid trajectories and a step-wise normalized trajectory representation to adapt this diverse data. In the RL stage, we present Adaptive Diversity-Aware Sampling (ADAS) that prioritizes high-diversity samples and introduce Spanning Driving Reward (SDR) with a focal style weighting to amplify reward's value span for improving sensitivity to driving quality. On the Navsim benchmark, Curious-VLA achieves SoTA results (PDMS 90.3, EPDMS 85.4) and a Best-of-N PDMS of 94.8, demonstrating its effectiveness in unlocking the exploratory potential of VLA models. Code: https://github.com/Mashiroln/curious_vla.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。