用教师学生框架让自动驾驶模型边推理边开车,提升准确率与可解释性。
Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving

- 教师模型通过反思式推理生成逻辑解释并优化,无需人工标注
- 学生模型蒸馏教师推理能力,结合路径解码器实现精准驾驶轨迹
- 在Waymo上零样本推理性能提升24%,适合追求可解释自动驾驶的开发者
端到端自动驾驶旨在从视觉输入直接映射到控制动作,但这类模型常为黑箱,难以应对复杂场景。近期工作引入视觉语言模型(VLM)以提供显式推理,提升可解释性与鲁棒性,但通常依赖预生成标注,存在标签错误且成本高昂。本文提出一种教师-学生架构,融合结构化推理与几何精度。教师模型通过反思式推理,由VLM生成逻辑解释,并在真实动作监督下自我修正,增强零样本泛化能力。学生模型通过监督微调蒸馏教师的推理能力,同时设计独立的路径点解码器,将文本推理转化为连续轨迹。该框架兼顾可解释性与驾驶性能,在Waymo基准测试中,相较基线模型在零样本推理、路径点准确率和推理效率上均表现更优。实验表明,加入推理文本使性能提升约24%。本工作推动了可解释、可部署的推理驱动自动驾驶系统发展。
原文摘要 · Abstract (English)
End-to-end (E2E) autonomous driving aims to learn a direct mapping from visual observations to control actions. However, these E2E models often act as black boxes and struggle with complex scenarios. To address this, recent works incorporate Vision-Language Models (VLMs) to provide explicit reasoning, enhancing both interpretability and driving robustness. These approaches typically rely on pre-generated annotations, which suffer from potentially flawed labels and require costly human labor. In this work, we propose a new framework that integrates structured reasoning and geometric precision through a teacher-student architecture. The teacher model introduces reflective reasoning, where the VLM generates logical explanations and then reflectively refines the reasoning under the supervision of ground-truth action. This enhances zero-shot generalization without intermediate labels. The student model distills the teacher's reasoning capabilities via supervised fine-tuning. We also design a separate waypoint decoder that interprets textual reasoning into continuous trajectories. Our proposed solution integrates two goals: providing explicit reasoning for interpretability and delivering robust and accurate driving performance. It leverages the synergy between these two objectives within a staged inference engine to enhance driving performance and explicitly uses the reasoning to guide driving prediction. Evaluated on Waymo benchmarks, our framework outperforms classical reasoning-based baselines in zero-shot reasoning, waypoint accuracy, and inference efficiency. Our experiments validate this design, demonstrating that the reasoning text makes a significant contribution to driving inference, resulting in around a 24% improvement in performance compared to an identical model that lacks reasoning. Our work advances reasoning-driven autonomous driving toward interpretable and deployable systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。