用强化学习约束扩散模型，让自动驾驶更安全多样。
DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving
- 引入自适应噪声与分层强化学习，控制生成轨迹质量。
- 在两个数据集上分别达到91.2和85.5的高分表现。
- 适合关注自动驾驶生成多样性与稳定性的研究者。
面向端到端自动驾驶的生成式扩散模型常因模式崩溃导致行为保守且同质化。尽管DiffusionDrive通过预设锚点划分动作空间以生成多样化轨迹，但其依赖模仿学习缺乏足够约束，难以兼顾多样性与高质量输出。本文提出DiffusionDriveV2，利用强化学习同时约束低质量模式并探索更优轨迹，在保持核心高斯混合模型多模态特性基础上显著提升整体输出质量。首先采用尺度自适应乘性噪声，促进轨迹规划中的广泛探索；其次引入锚内GRPO处理单个锚点样本的优势估计，以及锚间截断GRPO实现跨锚点全局视角，避免不同驾驶意图（如转弯与直行）间的不当优势比较，防止进一步模式崩溃。在闭环评估中，使用对齐的ResNet-34骨干网络，DiffusionDriveV2在NAVSIM v1数据集上达到91.2 PDMS，NAVSIM v2数据集上达到85.5 EPDMS，创下新纪录。实验验证该方法有效解决截断扩散模型中多样性与一致高质量之间的矛盾，实现了最佳权衡。代码与模型将公开于https://github.com/hustvl/DiffusionDriveV2。
原文摘要 · Abstract (English)
Generative diffusion models for end-to-end autonomous driving often suffer from mode collapse, tending to generate conservative and homogeneous behaviors. While DiffusionDrive employs predefined anchors representing different driving intentions to partition the action space and generate diverse trajectories, its reliance on imitation learning lacks sufficient constraints, resulting in a dilemma between diversity and consistent high quality. In this work, we propose DiffusionDriveV2, which leverages reinforcement learning to both constrain low-quality modes and explore for superior trajectories. This significantly enhances the overall output quality while preserving the inherent multimodality of its core Gaussian Mixture Model. First, we use scale-adaptive multiplicative noise, ideal for trajectory planning, to promote broad exploration. Second, we employ intra-anchor GRPO to manage advantage estimation among samples generated from a single anchor, and inter-anchor truncated GRPO to incorporate a global perspective across different anchors, preventing improper advantage comparisons between distinct intentions (e.g., turning vs. going straight), which can lead to further mode collapse. DiffusionDriveV2 achieves 91.2 PDMS on the NAVSIM v1 dataset and 85.5 EPDMS on the NAVSIM v2 dataset in closed-loop evaluation with an aligned ResNet-34 backbone, setting a new record. Further experiments validate that our approach resolves the dilemma between diversity and consistent high quality for truncated diffusion models, achieving the best trade-off. Code and model will be available at https://github.com/hustvl/DiffusionDriveV2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。