arXiv:2503.00502cs.RO2025-03被引 7

用大模型让自动驾驶车能实时与人类车互动,提升安全性和效率。

Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions

  • 构建并行推理-执行框架,通过记忆库实现与不同人类车的双向交互。
  • 在多场景测试中,安全性和决策效率显著优于传统方法。
  • 适合关注自动驾驶交互、人机协同的工程师和研究人员。

自动驾驶汽车已进入商业化阶段,但其与人类驾驶车辆的交互能力有限,难以有效表达意图。大语言模型(LLM)虽支持双向人机通信,但推理速度慢与实时决策需求存在冲突。为此,本文提出一种并行的演员-推理框架,通过训练时让基于LLM的推理器与异构模拟人类车交互,建立称为“演员”的交互记忆数据库。引入记忆分区模块和两层记忆检索机制后,演员对多样化人类车的适应能力显著增强。消融实验及与其他决策方法对比表明,该框架在多场景下显著提升安全性与效率。结合推理器生成的外部人机界面(eHMI)信息与演员检索到的可行动作方案,进一步验证了框架在真实多场景交互中的有效性。代码已开源:https://github.com/FanGShiYuu/Actor-Reasoner。

原文摘要 · Abstract (English)

Autonomous Vehicles (AVs) have entered the commercialization stage, but their limited ability to interact and express intentions still poses challenges in interactions with Human-driven Vehicles (HVs). Recent advances in large language models (LLMs) enable bidirectional human-machine communication, but the conflict between slow inference speed and the need for real-time decision-making challenges practical deployment. To address these issues, this paper introduces a parallel Actor-Reasoner framework designed to enable explicit bidirectional AV-HV interactions across multiple scenarios. First, by facilitating interactions between the LLM-driven Reasoner and heterogeneous simulated HVs during training, an interaction memory database, referred to as the Actor, is established. Then, by introducing the memory partition module and the two-layer memory retrieval module, the Actor's ability to handle heterogeneous HVs is significantly enhanced. Ablation studies and comparisons with other decision-making methods demonstrate that the proposed Actor-Reasoner framework significantly improves safety and efficiency. Finally, with the combination of the external Human-Machine Interface (eHMI) information derived from Reasoner's reasoning and the feasible action solutions retrieved from the Actor, the effectiveness of the proposed Actor-Reasoner is confirmed in multi-scenario field interactions. Our code is available at https://github.com/FanGShiYuu/Actor-Reasoner.

自动驾驶大模型人机交互决策框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。