用知识蒸馏让小模型学会大模型的驾驶规划能力。
On-Policy Distillation of Language Models for Autonomous Vehicle Motion Planning
- 让小模型自生成轨迹,用大模型逐字反馈来训练。
- 模型缩小五倍仍接近大模型性能,显著优于强化学习基线。
- 适合想在车载系统部署大模型规划能力的研究者。
大型语言模型(LLMs)最近在自动驾驶运动规划中展现出强大潜力,通过将轨迹预测重新表述为语言生成问题。然而,在资源受限的车载系统中部署高性能的LLM仍面临根本性挑战。本文研究如何有效将大教师模型的运动规划知识迁移至更小、更易部署的学生模型。基于GPT-Driver框架——该框架将驾驶场景表示为语言提示,并通过思维链推理生成航点轨迹——我们探究了两种学生训练范式:(i) 在策略广义知识蒸馏(GKD),即学生在自身生成输出上训练,利用教师提供的密集标记级反馈;(ii) 密集反馈强化学习(RL)基线,使用教师的对数概率作为每标记奖励信号,在策略梯度框架中训练。在nuScenes基准上的实验表明,GKD显著优于RL基线,并在模型规模缩小5倍的情况下,几乎达到教师模型性能。这些结果凸显了在策略蒸馏作为部署基于LLM规划器的一种原则性且高效的方法的实际价值。
原文摘要 · Abstract (English)
Large language models (LLMs) have recently demonstrated strong potential for autonomous vehicle motion planning by reformulating trajectory prediction as a language generation problem. However, deploying capable LLMs in resource-constrained onboard systems remains a fundamental challenge. In this paper, we study how to effectively transfer motion planning knowledge from a large teacher LLM to a smaller, more deployable student model. We build on the GPT-Driver framework, which represents driving scenes as language prompts and generates waypoint trajectories with chain-of-thought reasoning, and investigate two student training paradigms: (i) on-policy generalized knowledge distillation (GKD), which trains the student on its own self-generated outputs using dense token-level feedback from the teacher, and (ii) a dense-feedback reinforcement learning (RL) baseline that uses the teacher's log-probabilities as per-token reward signals in a policy gradient framework. Experiments on the nuScenes benchmark show that GKD substantially outperforms the RL baseline and closely approaches teacher-level performance despite a 5$\times$ reduction in model size. These results highlight the practical value of on-policy distillation as a principled and effective approach to deploying LLM-based planners in autonomous driving systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。