用激光雷达教摄像头更好识别3D物体,精度达62.8%
TiGDistill-BEV: Multi-view BEV 3D Object Detection via Target Inner-Geometry Learning Distillation
- 通过目标内几何学习,融合深度与鸟瞰特征提升检测能力
- 在nuScenes上实现62.8% NDS,显著超越现有纯摄像头方法
- 适合做自动驾驶视觉感知的工程师和研究者参考
准确的多视角3D目标检测对自动驾驶等应用至关重要。现有方法常利用激光雷达的精确空间信息,通过深度监督或鸟瞰图(BEV)特征蒸馏来增强基于相机的检测器,但因激光雷达与相机数据表征差异大而面临挑战。本文提出TiGDistill-BEV,一种新方法,通过目标内几何学习蒸馏,有效弥合模态差距。该方法将多模态(如激光雷达)作为教师模型,指导基于相机的学生检测器,引入两个关键模块:内深度监督模块用于学习物体内部相对深度关系,增强对物体空间结构的理解;内特征BEV蒸馏模块用于传递前景目标中关键点的高层语义。为缓解域差距,进一步采用通道间与关键点间蒸馏建模特征相似性。在nuScenes基准上的大量实验表明,TiGDistill-BEV显著提升纯摄像头检测器性能,达到62.8% NDS,超越此前方法。代码已公开于https://github.com/Public-BOTs/TiGDistill-BEV.git。
原文摘要 · Abstract (English)
Accurate multi-view 3D object detection is essential for applications such as autonomous driving. Researchers have consistently aimed to leverage LiDAR's precise spatial information to enhance camera-based detectors through methods like depth supervision and bird-eye-view (BEV) feature distillation. However, existing approaches often face challenges due to the inherent differences between LiDAR and camera data representations. In this paper, we introduce the TiGDistill-BEV, a novel approach that effectively bridges this gap by leveraging the strengths of both sensors. Our method distills knowledge from diverse modalities(e.g., LiDAR) as the teacher model to a camera-based student detector, utilizing the Target Inner-Geometry learning scheme to enhance camera-based BEV detectors through both depth and BEV features by leveraging diverse modalities. Specially, we propose two key modules: an inner-depth supervision module to learn the low-level relative depth relations within objects which equips detectors with a deeper understanding of object-level spatial structures, and an inner-feature BEV distillation module to transfer high-level semantics of different key points within foreground targets. To further alleviate the domain gap, we incorporate both inter-channel and inter-keypoint distillation to model feature similarity. Extensive experiments on the nuScenes benchmark demonstrate that TiGDistill-BEV significantly boosts camera-based only detectors achieving a state-of-the-art with 62.8% NDS and surpassing previous methods by a significant margin. The codes is available at: https://github.com/Public-BOTs/TiGDistill-BEV.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。