用视觉语言模型自动对齐自动驾驶预测与人类驾驶偏好。
VL-DPO: Vision-Language-Guided Finetuning for Preference-Aligned Autonomous Driving
- 用VLM自动生成人类偏好对比数据，指导模型微调。
- 在Waymo数据集上提升11.94%偏好评分，降低10.01%轨迹误差。
- 适合关注自动驾驶人性化决策与偏好学习的研究者。
自动驾驶数据集的快速增长推动了强大运动预测模型的发展。尽管大规模预训练能带来优异性能，但标准模仿学习目标难以捕捉人类驾驶偏好的复杂细节。近期视觉语言模型（VLM）在推理与常识理解方面表现突出。本文提出VL-DPO，一种基于视觉语言引导的框架，将自车运动预测模型与人类偏好对齐。该方法利用VLM作为零样本推理器，从预训练模型的轨迹回放中自动生成偏好对，并通过直接偏好优化（DPO）进行微调。我们在Waymo开放端到端驾驶数据集（WOD-E2E）上训练模型，并使用评审反馈分数（RFS）和平均位移误差（ADE）评估其性能。实验表明，VLM生成的轨迹选择是人类偏好的高质量代理。最终模型VL-DPO相较预训练模型在RFS上提升11.94%，在ADE上降低10.01%。
原文摘要 · Abstract (English)
The rapid growth of autonomous driving datasets has enabled the scaling of powerful motion forecasting models. While large-scale pretraining provides strong performance, the standard imitation objective may not fully capture the complex nuances of human driving preferences. Meanwhile, recent advances in vision-language models (VLMs) have demonstrated impressive reasoning and commonsense understanding. Building on these capabilities, this paper presents VL-DPO, a vision-language-guided framework that aligns ego-vehicle motion forecasting models with human preferences. Our approach leverages a VLM as a zero-shot reasoner to automatically generate preference pairs from a pretrained model's rollouts, which are then used to finetune the model via Direct Preference Optimization (DPO). We finetune our models on the Waymo Open End-to-End Driving Dataset (WOD-E2E) and evaluate performance against held-out human preference annotations using rater feedback score (RFS) and average displacement error (ADE). Our experiments confirm that the VLM's trajectory selection is a high-quality proxy for human preference. Our final model, VL-DPO, yields an 11.94% increase in RFS and a 10.01% reduction in ADE over the pretrained model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。