用轨迹引导世界模型,让自动驾驶模型跨域通用。
HEAT: Heterogeneous End-to-End Autonomous Driving via Trajectory-Guided World Models

- 以规划轨迹为中心组织训练,提取驾驶意图的域不变表征。
- 在nuScenes、NAVSIM和Waymo数据集上性能均显著提升。
- 适合需要跨城市、多传感器部署的自动驾驶系统研发者。
端到端自动驾驶通过直接将原始传感器数据映射为驾驶动作,成为传统模块化流程的有力替代。尽管现有方法在单一数据集上表现良好,但在跨多个异构域联合训练时性能显著下降。现实中,自动驾驶系统需在不同城市、传感器配置和交通模式等异构环境中运行,且无需针对每个域重新训练。这一差距凸显了多域学习的核心挑战:异构域间的域特定差异引入冲突学习信号,导致模型趋向次优的折中解。为此,我们提出一种基于轨迹驱动的学习范式,围绕规划轨迹组织训练,使模型能够捕捉驾驶意图的域不变表征。此外,引入一个世界模型,预测以自身行为为条件的未来潜在特征,增强特征一致性并缓解域相关偏差。我们在nuScenes、NAVSIM和Waymo端到端数据集上评估该方法,结果表明其在所有域中均显著优于现有方法。实验显示,仅用一个统一模型在异构数据上训练,即可在各域内保持强性能,迈向可扩展的真实世界部署。代码将公开。
原文摘要 · Abstract (English)
End-to-end autonomous driving has emerged as a compelling alternative to traditional modular pipelines by directly mapping raw sensor data to driving actions. While recent approaches achieve strong performance on single-domain datasets, their performance degrades significantly when trained jointly across multiple heterogeneous domains. In practice, however, autonomous systems must operate across diverse environments with heterogeneous distributions, including different cities, sensor configurations, and traffic patterns, without domain-specific retraining. This gap highlights a key challenge in multi-domain learning: domain-specific variations across heterogeneous domains introduce conflicting learning signals, driving models toward compromised solutions that are suboptimal across domains. To address this, we propose a trajectory-driven learning paradigm that organizes training around planning trajectories, enabling the model to capture domain-invariant representations of driving intent. Furthermore, we incorporate a world model that predicts future latent features conditioned on ego actions, improving feature consistency and mitigating domain-induced biases. We evaluate our approach on three benchmarks, nuScenes, NAVSIM, and the Waymo end-to-end dataset, and show substantial improvements over existing methods across all domains. Our results demonstrate that a single unified model can be trained on heterogeneous datasets while maintaining strong performance within each domain, highlighting a step toward scalable real-world deployment. We will make our code publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。