arXiv:2603.07464cs.CV2026-03被引 16

解决单目3D检测中跨模态蒸馏的负迁移问题,提升深度信息传递效果。

Selective Transfer Learning of Cross-Modality Distillation for Monocular 3D Object Detection

  • 设计选择性蒸馏机制,仅传递有效深度特征与关系。
  • 在KITTI和NuScenes上显著提升检测精度,优于现有SOTA模型。
  • 适用于CNN与DETR类模型,适配自动驾驶场景的单目感知系统。

单目3D目标检测因缺乏精确深度信息而面临严重挑战。跨模态知识蒸馏可将激光雷达(LiDAR)中的深度信息迁移到图像网络,但图像与激光雷达之间的模态差异严重限制了其性能。本文首次系统研究该任务中模态差距引发的负迁移问题,包括结构不一致与特征过拟合两大因素。为此提出名为MonoSTL的选择性学习方法,通过相似网络结构保证特征空间对齐,并设计两个新蒸馏模块:深度感知选择性特征蒸馏(DASFD)与深度感知选择性关系蒸馏(DASRD),分别结合深度不确定性实现对正向特征与关系的有选择性学习。该方法可无缝集成于多种基于CNN与DETR的模型中,在KITTI上验证了三种近期模型及NuScenes上一种模型,实验表明其显著提升基线模型精度,超越所有近期SOTA方法。

原文摘要 · Abstract (English)

Monocular 3D object detection is a promising yet ill-posed task for autonomous vehicles due to the lack of accurate depth information. Cross-modality knowledge distillation could effectively transfer depth information from LiDAR to image-based network. However, modality gap between image and LiDAR seriously limits its accuracy. In this paper, we systematically investigate the negative transfer problem induced by modality gap in cross-modality distillation for the first time, including not only the architecture inconsistency issue but more importantly the feature overfitting issue. We propose a selective learning approach named MonoSTL to overcome these issues, which encourages positive transfer of depth information from LiDAR while alleviates the negative transfer on image-based network. On the one hand, we utilize similar architectures to ensure spatial alignment of features between image-based and LiDAR-based networks. On the other hand, we develop two novel distillation modules, namely Depth-Aware Selective Feature Distillation (DASFD) and Depth-Aware Selective Relation Distillation (DASRD), which selectively learn positive features and relationships of objects by integrating depth uncertainty into feature and relation distillations, respectively. Our approach can be seamlessly integrated into various CNN-based and DETR-based models, where we take three recent models on KITTI and a recent model on NuScenes for validation. Extensive experiments show that our approach considerably improves the accuracy of the base models and thereby achieves the best accuracy compared with all recently released SOTA models.

3D检测跨模态蒸馏自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。