arXiv:2606.12236cs.ROcs.CV2026-06

用智能代理自动设计与调度自动驾驶系统,提升效率与实时性。

DrivingAgent: Design and Scheduling Agents for Autonomous Driving Systems

论文配图:DrivingAgent: Design and Scheduling Agents for Autonomous Driving Systems
图 1 · 摘自论文原文
  • 用大模型自动生成模块代码并验证,替代人工设计。
  • 在nuScenes和Bench2Drive上实现更优的快慢-精度平衡。
  • 适合研究自动驾驶系统自动化与实时调度的团队。

越来越多的自动驾驶系统开始引入基础模型以增强泛化能力并应对长尾场景,但这一趋势带来了两大挑战:(i) 模块设计与集成过程繁琐且依赖人工;(ii) 缺乏满足严格实时性要求的智能动态调度机制。尽管大语言模型(LLM)为基础的智能体为自动化提供了可能,现有框架在自动驾驶场景中表现不佳,主要问题在于未能区分系统设计与实时调度的本质差异,将模块视为黑箱,且不支持持续运行。为此,我们提出DrivingAgent,一种专为自动驾驶系统设计与调度双重挑战定制的新型智能体框架。在设计阶段,DrivingAgent通过解析系统架构,自动生成代码并利用超网络训练进行模块验证;在调度阶段,它采用轻量级强化学习训练的LLM,结合结构化记忆(融合长期存储与带时间戳的短期上下文),动态编排系统模块。实验表明,DrivingAgent在nuScenes和Bench2Drive基准上均实现了更优的速度-精度权衡。

原文摘要 · Abstract (English)

Many autonomous driving systems are increasingly incorporating foundation models to improve generalization and handle long-tail scenarios. However, this trend introduces two key challenges: (i) the manual and labor-intensive process of designing and integrating new models, and (ii) the lack of intelligent, dynamic scheduling mechanisms to meet strict real-time constraints. While Large Language Model (LLM)-based agents offer a promising avenue for automation, existing frameworks are ill-suited for autonomous driving. Specifically, they fail to distinguish between the fundamentally different requirements of system design and real-time scheduling, treat modules as opaque black boxes, and are not designed for continuous operation. To address these limitations, we propose DrivingAgent, a novel agent framework tailored to the dual challenges of autonomous driving system design and scheduling. In the design phase, DrivingAgent automates module development by interpreting system architecture, generating code, and validating modules via super-network training. In the scheduling phase, it employs a lightweight LLM trained with reinforcement learning to dynamically orchestrate system modules in real time, supported by a structured memory that integrates long-term storage with timestamped short-term context. Experimental results demonstrate that DrivingAgent achieves a superior speed--accuracy trade-off on both the nuScenes and Bench2Drive benchmarks.

自动驾驶智能体调度LLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。