让自动驾驶仿真中的车辆模仿人类驾驶风格,支持多种风格自由切换。
PersonaDrive: Human-Style Retrieval-Augmented VLA Agents for Closed-Loop Driving Simulation

- 用人类驾驶示范数据,通过检索方式让模型学习不同驾驶风格。
- 在基准测试中,各风格表现均优于现有方法,最差风格仍超最强基线5.4%。
- 无需重新训练即可切换驾驶风格,适合需要多样化交通模拟的场景。
封闭回路自动驾驶仿真通常使用行为模式单一的非自我车辆代理,其行为由规则引擎或单一模式学习模型生成。近期工作通过观察数据后处理标签或大模型推断奖励权重引入风格差异,但这些信号仅为风格应奖励内容的代理,而非人类明确按特定风格驾驶的示范。我们提出PersonaDrive,一个将视觉-语言-动作(VLA)驾驶代理基于风格指令的人类驾驶数据检索示范进行条件化的新流程。该数据集在驾驶员在环系统中,按激进、中性、保守指令驾驶CARLA排行榜路线采集而成。流程包含三个阶段:(i) 利用图像-文本联合相似度分数,在各风格人类驾驶数据中离线挖掘三元组;(ii) 训练轻量级检索头,融合冻结视觉特征与小型控制编码器,构建各风格数据库;(iii) 微调单个VLA主干网络,将其检索到的上下文点作为推理时的上下文示范用于路径点预测。推理时,仅需更换检索头查询的风格数据库即可实现风格切换,无需为每种风格重新训练,从而在封闭回路仿真中实现无需重训练的、人类风格多样化的非自我车辆代理。在Bench2Drive上,无风格条件下的PersonaDrive相比SimLingo提升驾驶得分4.6%,相比HiP-AD提升2.5%;在风格条件控制下,所有风格表现均位列第一,误差在约2%范围内(最弱风格仍比最强基线DMW高出5.4%),平均速度与加速度从保守到激进指令分别提升18%和25%。
原文摘要 · Abstract (English)
Closed-loop driving simulators typically populate their environments with non-ego traffic agents that behave largely the same way, produced either by rule-based traffic managers or by learned models trained toward a single behavioral mode. Recent work introduces style variation through post-hoc labels on observational data or LLM-inferred reward weights, but these signals act as proxies for what a style should reward rather than demonstrations of humans explicitly asked to drive in that style. We introduce PersonaDrive, a pipeline that conditions a vision-language-action (VLA) driving agent on retrieved demonstrations from a style-instructed human driving dataset, in which participants drive CARLA leaderboard routes under aggressive, neutral, and conservative instructions on a driver-in-the-loop rig. The pipeline has three stages: (i) offline triplet mining over per-style human driving data using a combined image-text similarity score; (ii) training a lightweight retrieval head that fuses frozen visual features with a small control encoder over per-style databases; and (iii) fine-tuning a single VLA backbone to treat retrieved context points as in-context behavioral demonstrations during waypoint prediction. At inference, the same backbone is conditioned on any style by swapping which per-style database the retrieval head queries, so selecting a style requires no per-style retraining while enabling human-style, style-diverse non-ego agents for closed-loop simulation. On Bench2Drive, PersonaDrive (no style) improves the driving score by 4.6% over SimLingo and 2.5% over HiP-AD, and under style conditioning attains the highest driving score in every style within a roughly 2% band (its weakest style surpassing the strongest baseline, DMW, by 5.4%), while average speed and acceleration rise by 18% and 25% from the conservative to the aggressive instruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。