用流匹配加速生成策略,实现自动驾驶实时决策。
Real-Time Generative Policy via Langevin-Guided Flow Matching for Autonomous Driving
- 引入流匹配与朗之万动力学,单步生成最优动作。
- 在多车道和交叉口场景中性能超越基线,推理延迟极低。
- 适用于需要快速响应的自动驾驶系统,尤其适合高复杂度环境。
强化学习是自动驾驶系统的核心方法,生成式策略因其能建模复杂分布而具备强大探索能力。但其固有的高推理延迟严重阻碍了实时决策部署。为此,我们提出基于流匹配的扩散演员-评论家算法(DACER-F),将流匹配引入在线强化学习,可在单次推断中生成具有竞争力的动作。通过结合朗之万动力学与Q函数梯度,DACER-F动态优化经验回放中的动作,使其逼近兼顾高Q值与探索性的目标分布。随后,流策略被训练以高效地从简单先验分布映射到该动态目标。在复杂的多车道及交叉口仿真中,DACER-F优于基线方法(如DACER与DSAC),同时保持极低推理延迟。该方法还在标准基准DeepMind Control Suite(DMC)上展现良好可扩展性,在humanoid-stand任务中取得775.8分,超越已有方法。结果表明,DACER-F是一种高性能且计算高效的强化学习算法。
原文摘要 · Abstract (English)
Reinforcement learning (RL) is a fundamental methodology in autonomous driving systems, where generative policies exhibit considerable potential by leveraging their ability to model complex distributions to enhance exploration. However, their inherent high inference latency severely impedes their deployment in real-time decision-making and control. To address this issue, we propose diffusion actor-critic with entropy regulator via flow matching (DACER-F) by introducing flow matching into online RL, enabling the generation of competitive actions in a single inference step. By leveraging Langevin dynamics and gradients of the Q-function, DACER-F dynamically optimizes actions from experience replay toward a target distribution that balances high Q-value information with exploratory behavior. The flow policy is then trained to efficiently learn a mapping from a simple prior distribution to this dynamic target. In complex multi-lane and intersection simulations, DACER-F outperforms baselines diffusion actor-critic with entropy regulator (DACER) and distributional soft actor-critic (DSAC), while maintaining an ultra-low inference latency. DACER-F further demonstrates its scalability on standard RL benchmark DeepMind Control Suite (DMC), achieving a score of 775.8 in the humanoid-stand task and surpassing prior methods. Collectively, these results establish DACER-F as a high-performance and computationally efficient RL algorithm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。