让自动驾驶模型同时学专家动作和自我优化，提升罕见场景表现
CoIRL-AD: Collaborative-Competitive Imitation-Reinforcement Learning in Latent World Models for Autonomous Driving
- 用双策略分离模仿与奖励学习，避免目标冲突
- 通过想象未来轨迹评估长期奖励，提升决策能力
- 适合需要强泛化能力的自动驾驶研发团队
端到端自动驾驶模型采用模仿学习（IL）训练时，在长尾场景中泛化能力差，因专家示范稀疏。强化学习（RL）可提供任务级监督，但在无交互模拟器的离线设置下难以应用，因数据集以专家行为为主，缺乏行为多样性。本文提出CoIRL-AD，一种基于潜在世界模型的协同-竞争双策略框架，在统一离线训练范式下融合IL与RL。该方法将模仿与奖励优化解耦为独立智能体，利用想象的未来轨迹进行长周期奖励估计，并引入竞争机制，选择性迁移有益行为，同时保持RL策略贴近专家驾驶风格。在nuScenes基准上的实验表明，CoIRL-AD在跨城市泛化和长尾场景中显著优于强基线模型，性能持续提升。代码已开源：https://github.com/SEU-zxj/CoIRL-AD。
原文摘要 · Abstract (English)
End-to-end autonomous driving models trained with imitation learning (IL) often generalize poorly, particularly in long-tail scenarios where expert demonstrations are sparse. Reinforcement learning (RL) can provide complementary task-level supervision, but applying RL to real-world autonomous driving is challenging in offline settings without interactive simulators, where datasets are dominated by expert actions and provide limited behavioral diversity. We propose CoIRL-AD, a competitive dual-policy framework that integrates IL and RL under a unified offline training regime. CoIRL-AD decouples imitation and reward optimization into separate actors to alleviate objective conflicts, uses imagined future rollouts for long-horizon reward estimation, and introduces a competition mechanism that selectively transfers beneficial behaviors while keeping RL anchored to expert-like driving. Experiments on the nuScenes benchmark show that CoIRL-AD consistently improves robustness over strong IL-based baselines, with especially large gains in cross-city generalization and long-tail scenarios. Code is available at: https://github.com/SEU-zxj/CoIRL-AD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。