arXiv:2605.16858cs.ROcs.AI2026-05中稿 · presentation at th…

用大模型理解行人行为,让自动驾驶更安全地应对突发状况。

Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles

论文配图:Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles
图 1 · 摘自论文原文
  • 将环境信息转为自然语言提示,让大模型推理行人意图并生成谨慎驾驶策略。
  • 零样本下碰撞率降低至32%,在突发过街等场景中表现远超传统强化学习方法。
  • 决策过程可解释,适合对安全性与透明度要求高的自动驾驶应用。

自动驾驶车辆在密集城市环境中需应对多变甚至异常的行人行为,而基于强化学习的控制系统在非结构化交通中泛化能力差,依赖人工设计奖励函数且决策不透明。为此,本文提出一种基于大语言模型(LLM)的行人感知行为规划框架。该系统将结构化场景观测转化为自然语言推理提示,使LLM能够推断行人意图、预判风险并生成保守的驾驶策略;这些策略由运动规划器执行,确保轨迹平滑且符合动力学约束。在SUMO仿真中评估多种行人交互场景,包括意外穿行、折返过街、犹豫和双向过街。零样本测试中,该框架实现68%的无碰撞成功率,显著优于深度强化学习基线(17.7%)。在单行人场景中引入少量经验记忆后,性能提升至96.0%,超过定制DQN控制器(82.0%)。跨行为评估显示,从折返交互中学习的记忆可迁移至未见过的犹豫和双向过街场景,成功率分别为82.0%和90.0%。系统始终提前响应,保持更大安全距离,并生成可解释的人类对齐决策。

原文摘要 · Abstract (English)

Autonomous Vehicles (AVs) must make reliable decisions in dense urban environments where pedestrian behavior is variable, sometimes abnormal, and often unseen during training. Reinforcement learning (RL)-based AV control systems perform well in structured traffic but struggle to generalize to unpredictable pedestrian interactions and out-of-distribution scenarios. Their reliance on handcrafted rewards and opaque decisions further limits their suitability for safety-critical, pedestrian-rich environments. To address these limitations, we introduce a Large Language Model (LLM)-based decision-making framework for pedestrian-aware behavioral planning. The system converts structured scene observations into natural-language reasoning prompts, enabling the LLM to infer pedestrian intent, anticipate risk, and generate cautious tactical driving decisions. These decisions are executed by a motion planner that ensures smooth, kinematically feasible control. We evaluate the framework in SUMO across multiple pedestrian-interaction scenarios, including unexpected jaywalking, turn-back crossing, hesitation, and bidirectional crossing. In zero-shot evaluation, the LLM-based agent achieves a 68% collision-free success rate, substantially outperforming deep RL baselines (17.7%). With few-shot episodic memory in a single-pedestrian scenario, performance increases to 96.0%, exceeding a custom DQN controller (82.0%). Cross-behavior evaluation further shows that memory derived from turn-back interactions transfers to unseen hesitation and bidirectional crossing scenarios, achieving 82.0% and 90.0% success, respectively. The system consistently initiates earlier responses, maintains wider safety buffers, and produces interpretable, human-aligned decisions.

自动驾驶大模型行人预测可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。