用隐空间模型把自动驾驶强化学习采样速度提升80倍，同时保持视觉可解释性。
DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
- 通过递归多分辨率压缩，将扩散采样从100步减至1步。
- 在NavSim v2上实现87.7 EPDMS，性能达当前最优。
- 适合追求高效安全训练的自动驾驶强化学习研究者。
我们提出DreamerAD，首个基于隐空间世界模型的自动驾驶强化学习框架，将扩散采样从100步压缩至1步，实现80倍加速，同时保持视觉可解释性。在真实驾驶数据上训练强化学习策略成本高昂且存在安全风险。现有像素级扩散世界模型虽支持安全想象式训练，但多步扩散推理延迟高达2秒/帧，无法满足高频强化学习交互需求。本方法利用视频生成模型的去噪隐特征，通过三个关键机制：(1) 快速强制机制，通过递归多分辨率步长压缩降低采样复杂度；(2) 基于隐表示的自回归密集奖励模型，实现细粒度信用分配；(3) 高斯词汇采样用于GRPO，约束探索范围在物理合理轨迹内。DreamerAD在NavSim v2上取得87.7 EPDMS，确立当前最佳性能，证明隐空间强化学习在自动驾驶中的有效性。
原文摘要 · Abstract (English)
We introduce DreamerAD, the first latent world model framework that enables efficient reinforcement learning for autonomous driving by compressing diffusion sampling from 100 steps to 1 - achieving 80x speedup while maintaining visual interpretability. Training RL policies on real-world driving data incurs prohibitive costs and safety risks. While existing pixel-level diffusion world models enable safe imagination-based training, they suffer from multi-step diffusion inference latency (2s/frame) that prevents high-frequency RL interaction. Our approach leverages denoised latent features from video generation models through three key mechanisms: (1) shortcut forcing that reduces sampling complexity via recursive multi-resolution step compression, (2) an autoregressive dense reward model operating directly on latent representations for fine-grained credit assignment, and (3) Gaussian vocabulary sampling for GRPO that constrains exploration to physically plausible trajectories. DreamerAD achieves 87.7 EPDMS on NavSim v2, establishing state-of-the-art performance and demonstrating that latent-space RL is effective for autonomous driving.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。