用大模型代理打通自动驾驶路径规划测试全流程。
PlannerForge: LLM Agents for Scenario-Based Testing of Motion Planners in Autonomous Driving
- 构建端到端大模型代理框架，覆盖场景生成到评估全链路。
- 开源模型性能接近商用API，成本调优使成功率提至70.2%。
- 适合自动驾驶安全验证研究者和工程团队使用。
确保自动驾驶安全性是关键挑战。场景化测试虽系统化，但各环节（场景生成、检索、修改、ADS执行、结果分析）分散于独立工具，缺乏交互。大语言模型代理已在感知、规划、控制等子系统展现潜力，但尚无统一框架覆盖完整测试流程。本文提出PlannerForge，一个基于LLM代理的统一框架，拓展了从场景生成到ADS评估的全过程，并新增两个增强阶段：ADS增强与基准测试。我们在5种提示条件下，用10个现成的LLM对所有任务（生成、选择、修改、模块路由、规划器测试、增强）进行评估。最优任务得分在0.88至1.00之间；开源20-35B模型在多数任务上媲美商用API。Qwen3.6:35B在五项任务中三项表现相当。模块串联端到端时，保留83%（商用）/78%（开源）种子查询。其自然语言生成能力优于Scenario Factory 2.0（193 vs. 144可执行/200），实现92-96%请求的城市、道路和车辆属性。在排名1的选择上超越BM25（92.0% vs. 67.5%），在物理有效编辑上显著优于From-Words-to-Collisions（≥94% vs. 31%）。当N=400时，成本调优将规划成功率从50.4%提升至70.2%，碰撞率从19.0%降至8.4%，无需领域微调。
原文摘要 · Abstract (English)
Ensuring the safety of autonomous driving is a critical challenge. Scenario-based testing is a systematic process used to validate Autonomous Driving Systems (ADSs), but it remains a fragmented modular pipeline in which scenario generation, retrieval, modification, ADS execution, and results analysis are performed by separate tools with little interaction. Large Language Model (LLM) agents have shown promise across ADS sub-systems such as perception, planning, and control. However, no prior work covers the whole scenario-based testing pipeline for ADSs with a unified LLM-agent framework. We present PlannerForge, an LLM-agent framework that extends all scenario-based testing stages (from Scenario Generation to ADS Assessment) and adds two further LLM-enhanced stages: ADS Enhancement and ADS Benchmarking. We evaluate PlannerForge with 10 off-the-shelf LLMs across all tasks (Generation, Selection, Modification, Module Routing, Planner Testing, and Enhancement) under 5 prompt conditions. Best-per-task scores range from 0.88 to 1.00, and open-source 20-35B backends match commercial APIs on most tasks. Open-source models such as Qwen3.6:35B match commercial APIs on three of the five tasks. Chaining the modules end-to-end retains 83% / 78% of seed queries (commercial / open). It outperforms Scenario Factory 2.0 (Finkeldei et al., 2025) on natural-language generation (193 vs. 144 executable of 200) and realises 92-96% of requested city, road and vehicle attributes. It outperforms BM25 (Robertson and Zaragoza, 2009) at rank 1 selection (92.0% vs. 67.5%) and From-Words-to-Collisions (Gao et al., 2025) on physically valid edits (>=94% vs. 31%). At N=400, cost-tuning lifts planner success from 50.4% to 70.2% and cuts collisions from 19.0% to 8.4%, without domain-specific fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。