用简单方法实现2D视觉模型向3D LiDAR模型的知识迁移，无需伪语义图。
CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation
- 直接使用特征相似性损失+MLP投影头，避免复杂损失设计。
- 在低数据量下仍提升10% mIoU，优于现有方法。
- 新增占用预测任务增强3D空间推理能力，适合自动驾驶场景。
视觉基础模型（如DINO）推动了基于2D相机感知的范式转变，能提取通用特征支持多种下游任务。近期工作通过自监督跨模态知识蒸馏（KD）将这些泛化能力迁移到3D LiDAR模型中，但多依赖复杂损失、伪语义图或仅限于语义分割任务。本文提出CleverDistiller，一种自监督的2D到3D跨模态蒸馏框架，采用简单有效的设计：不依赖对比学习的复杂损失，而是结合直接特征相似性损失与MLP投影头，使3D网络学习复杂的语义依赖关系；关键在于无需伪语义图，可直接从视觉基础模型迁移知识而无需显式语义监督。此外，引入辅助自监督空间任务——占用预测，以增强通过蒸馏获得的语义知识的3D空间推理能力。在标准自动驾驶基准上进行的2D到3D KD实验表明，该方法在语义分割和3D目标检测（3DOD）上均达到最新水平，尤其在微调时仅用极少数据即提升最高达10%的mIoU，验证了其简单却高效的知识蒸馏策略的有效性。
原文摘要 · Abstract (English)
Vision foundation models (VFMs) such as DINO have led to a paradigm shift in 2D camera-based perception towards extracting generalized features to support many downstream tasks. Recent works introduce self-supervised cross-modal knowledge distillation (KD) as a way to transfer these powerful generalization capabilities into 3D LiDAR-based models. However, they either rely on highly complex distillation losses, pseudo-semantic maps, or limit KD to features useful for semantic segmentation only. In this work, we propose CleverDistiller, a self-supervised, cross-modal 2D-to-3D KD framework introducing a set of simple yet effective design choices: Unlike contrastive approaches relying on complex loss design choices, our method employs a direct feature similarity loss in combination with a multi layer perceptron (MLP) projection head to allow the 3D network to learn complex semantic dependencies throughout the projection. Crucially, our approach does not depend on pseudo-semantic maps, allowing for direct knowledge transfer from a VFM without explicit semantic supervision. Additionally, we introduce the auxiliary self-supervised spatial task of occupancy prediction to enhance the semantic knowledge, obtained from a VFM through KD, with 3D spatial reasoning capabilities. Experiments on standard autonomous driving benchmarks for 2D-to-3D KD demonstrate that CleverDistiller achieves state-of-the-art performance in both semantic segmentation and 3D object detection (3DOD) by up to 10% mIoU, especially when fine tuning on really low data amounts, showing the effectiveness of our simple yet powerful KD strategy
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。