用强化学习提升交通模拟的逼真度与可控性，解决闭环测试中的偏差问题。
RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation
- 先数据驱动模仿学习，再物理引擎强化微调，分阶段提升效果
- 新方法在多个指标上超越基线，实现更高真实感和可控性
- 适合自动驾驶系统闭环评估与高保真仿真研究者使用
在自动驾驶领域，实现闭环交通模拟中的逼真性与可控性仍是关键挑战。基于数据的方法虽能还原真实轨迹，但在闭环部署时易受协变量偏移影响，且简化动力学模型降低可靠性；而基于物理的模拟方法虽具备可靠可控的交互能力，却缺乏专家示范，导致逼真性不足。为此，我们提出一种双阶段以车辆为中心的模拟框架：首先在数据驱动模拟器中进行模仿学习预训练，以捕捉轨迹级真实性和路线级可控性；随后在物理模拟器中通过强化学习微调，提升风格级可控性并缓解协变量偏移。微调阶段提出RIFT——一种新的组相对强化学习策略，通过组相对评估所有候选模态，并采用代理目标实现稳定优化，有效增强风格级可控性、缓解偏移，同时保留模仿学习继承的轨迹级真实性和路线级可控性。大量实验表明，RIFT在提升交通模拟真实性和可控性的基础上，还揭示了当前自动驾驶系统在闭环评估中的局限性。
原文摘要 · Abstract (English)
Achieving both realism and controllability in closed-loop traffic simulation remains a key challenge in autonomous driving. Dataset-based methods reproduce realistic trajectories but suffer from covariate shift in closed-loop deployment, compounded by simplified dynamics models that further reduce reliability. Conversely, physics-based simulation methods enhance reliable and controllable closed-loop interactions but often lack expert demonstrations, compromising realism. To address these challenges, we introduce a dual-stage AV-centric simulation framework that conducts imitation learning pre-training in a data-driven simulator to capture trajectory-level realism and route-level controllability, followed by reinforcement learning fine-tuning in a physics-based simulator to enhance style-level controllability and mitigate covariate shift. In the fine-tuning stage, we propose RIFT, a novel group-relative RL fine-tuning strategy that evaluates all candidate modalities through group-relative formulation and employs a surrogate objective for stable optimization, enhancing style-level controllability and mitigating covariate shift while preserving the trajectory-level realism and route-level controllability inherited from IL pre-training. Extensive experiments demonstrate that RIFT improves realism and controllability in traffic simulation while simultaneously exposing the limitations of modern AV systems in closed-loop evaluation. Project Page: https://currychen77.github.io/RIFT/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。