让大模型决策时实时验证动作可行性，提升自动驾驶安全性
Reason--Imagine--Act: Closed-Loop LLM Decision Making with World Models for Autonomous Driving
- 用世界模型对大模型的行动意图进行短时模拟验证
- 在CARLA上实现80.05%路线完成率、0.20%碰撞率
- 适合研究闭环决策与安全验证的自动驾驶学者
大语言模型在自动驾驶中虽具潜力，但仅依赖语义决策可能导致物理不安全行为。现有方法或缺乏动态验证，或仅在离线流程中使用世界模型，导致语义意图与物理可行性脱节。本文提出闭环框架RIA，将大模型推理器与动作条件世界模型结合，在线验证动作可行性。每一步中，大模型生成动作模板与候选子动作，世界模型执行短时轨迹推演，安全评分器选择最安全可执行动作，并反馈至下一推理步骤。在统一的CARLA点目标协议下（1000个回合），RIA实现80.05%路线完成率、51.10%到达率、0.20%碰撞率。相比无需训练的基线（CARLA TM 和 MADA），RIA在核心闭环指标上持续领先。代码已开源：https://github.com/pku-smart-city/source_code/tree/main/RIA。
原文摘要 · Abstract (English)
Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。