让自动驾驶模型从失败中自我进化,提升规划精度。
FIRE-VLA: Failure-Informed Self-Evolution for Vision-Language-Action Models in Autonomous Driving

- 通过自蒸馏机制,将失败轨迹转化为改进策略的监督信号。
- 在nuScenes测试集上,平均路径误差降低至1.500米,失败率下降至11.20%。
- 适合追求高可靠性自动驾驶规划的开发者与研究者。
强化学习通过评估当前策略采样的轨迹来提升自动驾驶视觉-语言-动作(VLA)模型性能。组内相对策略优化(GRPO)基于每组轨迹间的奖励差异进行学习。当所有采样轨迹均表现不佳时,该相对信号虽可排序失败,却无法识别失败区域外的行为。为此,本文提出FIRE-VLA——一种失败感知的自演化框架,将未解决的失败转化为下一阶段策略的特权监督。低奖励、低多样性的轨迹组会触发从同一模型冻结初始副本的自蒸馏过程。教师与学生模型参数规模相同,仅教师可见未来轨迹,监督信号仅作用于学生生成的前缀部分,而GRPO对每组持续有效。更新后的策略作为下一轮教师,使故障分布随策略动态变化,无需依赖更大外部教师。从相同的Qwen2.5-VL-3B SFT检查点开始,对比实验保持学生采样与策略更新次数一致。在150个保留nuScenes场景的6,019个样本上,FIRE-VLA维持相近单样本规划能力,将G=4的平均L2误差从1.848米降至1.500米,评估持续失败率由13.03%降至11.20%。误差降低主要来自罕见严重轨迹的改善,而非普通轨迹的均匀优化。
原文摘要 · Abstract (English)
Reinforcement learning improves autonomous-driving vision-language-action (VLA) models by evaluating trajectories sampled from the current policy. Group relative policy optimization (GRPO) learns from reward differences within each rollout group. When all sampled trajectories are poor, this relative signal can rank failures without identifying behavior outside the failed region. We introduce FIRE-VLA, a failure-informed self-evolution framework that converts such unresolved failures into privileged supervision for the next policy. Low-reward, low-diversity groups trigger self-distillation from a frozen round-start copy of the same model. Teacher and student have the same parameter scale, but only the teacher observes the hidden future trajectory. Supervision follows the student's generated prefix and is restricted to answer tokens, while GRPO remains active for every group. The updated policy supplies the teacher for the next round, allowing the routed failure distribution to change with the policy without requiring a larger external teacher. Starting from the same Qwen2.5-VL-3B SFT checkpoint, the comparison matches student rollout and policy-update counts. On 6,019 examples from 150 held-out nuScenes scenes, FIRE-VLA retains comparable single-sample planning, reduces G=4 mean L2 from 1.848 to 1.500 m, and lowers evaluation-persistent failure prevalence from 13.03% to 11.20%. The reduction in mean error arises mainly from rare severe rollouts rather than uniform improvement across ordinary trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。