将语义与运动学习分离，提升自动驾驶端到端模型的感知与预测能力。
Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Driving
- 用独立的运动查询并行处理检测、跟踪与预测任务。
- 在nuScenes上实现感知、预测与规划全链路性能提升。
- 适合研究多任务自动驾驶系统与模型架构设计者。
环境感知及其随时间的变化包含语义和运动两类根本但异质的信息。以往的端到端自动驾驶方法将这两类信息统一编码为单一特征向量，但在引入运动相关任务（如预测与规划）后，会损害检测与跟踪性能，即出现负迁移现象。为此，本文提出神经-贝叶斯运动解码机制，一种新型并行检测、跟踪与预测方法，实现了语义与运动学习的分离。具体而言，采用一组可学习的运动查询，与检测及跟踪查询并行运行，共享一套递归更新的参考点。同时，通过交互式语义解码增强语义任务中的信息交换，促进正向迁移。在nuScenes数据集上基于UniAD和SparseDrive的实验验证了该分治融合方法的有效性，显著提升了感知、预测与规划的整体性能。代码已开源：https://github.com/shenyinzhe/DMAD。
原文摘要 · Abstract (English)
Perceiving the environment and its changes over time corresponds to two fundamental yet heterogeneous types of information: semantics and motion. Previous end-to-end autonomous driving works represent both types of information in a single feature vector. However, including motion related tasks, such as prediction and planning, impairs detection and tracking performance, a phenomenon known as negative transfer in multi-task learning. To address this issue, we propose Neural-Bayes motion decoding, a novel parallel detection, tracking, and prediction method that separates semantic and motion learning. Specifically, we employ a set of learned motion queries that operate in parallel with detection and tracking queries, sharing a unified set of recursively updated reference points. Moreover, we employ interactive semantic decoding to enhance information exchange in semantic tasks, promoting positive transfer. Experiments on the nuScenes dataset with UniAD and SparseDrive confirm the effectiveness of our divide and merge approach, resulting in performance improvements across perception, prediction, and planning. Our code is available at https://github.com/shenyinzhe/DMAD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。