arXiv:2511.07106cs.CV2025-11被引 5

提出混合编码与多任务学习框架,实现3D感知与端到端自动驾驶的高效协同。

HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Driving

  • 短时帧用大图像编码器,长时帧用小编码器,平衡计算资源与特征质量。
  • 同时提取稠密与稀疏特征,提升多任务精度,降低累积误差。
  • 在nuScenes上达到领先性能,碰撞率最低,适合自动驾驶系统集成。

三维特征提取是自动驾驶系统的关键组件,3D目标检测、鸟瞰图语义分割和占据预测等感知任务对三维特征提出重要约束。尽管大型图像编码器、高分辨率图像和长期时序输入能显著提升特征质量并带来显著性能提升,但这些技术在训练和推理中常因计算资源限制而难以兼容。此外,不同任务偏好不同的特征表示,导致单一模型在多任务端到端推理时难以维持与单任务模型相当的精度。为此,我们提出HENet和HENet++框架,用于多任务3D感知与端到端自动驾驶。具体地,我们设计了一种混合图像编码网络,对短时帧使用大型编码器,对长时帧使用小型编码器;同时,框架同步提取稠密与稀疏特征,为不同任务提供更适配的表示,减少累积误差,并向规划模块提供更全面的信息。所提架构兼容多种现有3D特征提取方法,支持多模态输入。HENet++在nuScenes基准上实现领先的端到端多任务3D感知结果,同时在nuScenes端到端自动驾驶基准上取得最低碰撞率。

原文摘要 · Abstract (English)

Three-dimensional feature extraction is a critical component of autonomous driving systems, where perception tasks such as 3D object detection, bird's-eye-view (BEV) semantic segmentation, and occupancy prediction serve as important constraints on 3D features. While large image encoders, high-resolution images, and long-term temporal inputs can significantly enhance feature quality and deliver remarkable performance gains, these techniques are often incompatible in both training and inference due to computational resource constraints. Moreover, different tasks favor distinct feature representations, making it difficult for a single model to perform end-to-end inference across multiple tasks while maintaining accuracy comparable to that of single-task models. To alleviate these issues, we present the HENet and HENet++ framework for multi-task 3D perception and end-to-end autonomous driving. Specifically, we propose a hybrid image encoding network that uses a large image encoder for short-term frames and a small one for long-term frames. Furthermore, our framework simultaneously extracts both dense and sparse features, providing more suitable representations for different tasks, reducing cumulative errors, and delivering more comprehensive information to the planning module. The proposed architecture maintains compatibility with various existing 3D feature extraction methods and supports multimodal inputs. HENet++ achieves state-of-the-art end-to-end multi-task 3D perception results on the nuScenes benchmark, while also attaining the lowest collision rate on the nuScenes end-to-end autonomous driving benchmark.

3D感知多任务学习自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。