提出统一框架DriveTransformer,实现端到端自动驾驶的高效协同与稳定训练。
DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving

- 任务并行设计,感知、规划等模块实时交互,消除误差累积
- 稀疏表示与流式处理,降低计算开销,支持长时序融合
- 在Bench2Drive和nuScenes上均达顶尖性能,适合大规模部署
端到端自动驾驶(E2E-AD)正成为自动驾驶领域的重要趋势,其数据驱动方法具有可扩展性优势。然而,现有方法多采用感知-预测-规划的串行范式,导致误差累积和训练不稳定;任务间手动排序也限制了协同潜力(如规划感知、博弈论交互预测与规划)。此外,密集的鸟瞰图(BEV)表示带来长距离感知与长期时序融合的计算挑战。为此,本文提出DriveTransformer,一种简化且易于扩展的E2E-AD框架,具备三大特性:任务并行(所有代理、地图与规划查询在每层直接交互)、稀疏表示(任务查询直接与原始传感器特征交互)、流式处理(任务查询作为历史信息存储传递)。由此形成三个统一操作:任务自注意力、传感器交叉注意力、时序交叉注意力,显著降低系统复杂度并提升训练稳定性。DriveTransformer在模拟闭环基准Bench2Drive与真实世界开环基准nuScenes上均取得领先性能,并实现高帧率运行。
原文摘要 · Abstract (English)
End-to-end autonomous driving (E2E-AD) has emerged as a trend in the field of autonomous driving, promising a data-driven, scalable approach to system design. However, existing E2E-AD methods usually adopt the sequential paradigm of perception-prediction-planning, which leads to cumulative errors and training instability. The manual ordering of tasks also limits the system`s ability to leverage synergies between tasks (for example, planning-aware perception and game-theoretic interactive prediction and planning). Moreover, the dense BEV representation adopted by existing methods brings computational challenges for long-range perception and long-term temporal fusion. To address these challenges, we present DriveTransformer, a simplified E2E-AD framework for the ease of scaling up, characterized by three key features: Task Parallelism (All agent, map, and planning queries direct interact with each other at each block), Sparse Representation (Task queries direct interact with raw sensor features), and Streaming Processing (Task queries are stored and passed as history information). As a result, the new framework is composed of three unified operations: task self-attention, sensor cross-attention, temporal cross-attention, which significantly reduces the complexity of system and leads to better training stability. DriveTransformer achieves state-of-the-art performance in both simulated closed-loop benchmark Bench2Drive and real world open-loop benchmark nuScenes with high FPS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。