用单目大模型蒸馏监督，提升稀疏点云深度补全精度
Distilling Monocular Foundation Model for Fine-grained Depth Completion
- 分两阶段蒸馏：先用自然图生成伪深度数据，再用不变尺度损失对齐真实尺度
- 在KITTI上达到榜首性能，显著优于现有方法
- 适合做自动驾驶深度感知的工程师和研究者
深度补全旨在从稀疏激光雷达输入中预测稠密深度图。然而，传感器提供的稀疏深度标注限制了稠密监督信号，难以学习精细几何特征。本文提出一种两阶段知识蒸馏框架，利用强大的单目基础模型提供稠密监督。第一阶段通过单目深度与网格重建模拟激光雷达扫描，生成无需真值深度的训练数据，实现几何知识迁移；第二阶段引入尺度与平移不变损失（SSI Loss），在真实数据集微调时学习真实尺度，解决单目深度固有的尺度模糊问题。该框架使深度补全模型有效利用单目基础模型的优势。实验表明，采用该方法训练的模型在KITTI基准上取得第一名成绩。代码已公开于 https://github.com/Sharpiless/DMD3C。
原文摘要 · Abstract (English)
Depth completion involves predicting dense depth maps from sparse LiDAR inputs. However, sparse depth annotations from sensors limit the availability of dense supervision, which is necessary for learning detailed geometric features. In this paper, we propose a two-stage knowledge distillation framework that leverages powerful monocular foundation models to provide dense supervision for depth completion. In the first stage, we introduce a pre-training strategy that generates diverse training data from natural images, which distills geometric knowledge to depth completion. Specifically, we simulate LiDAR scans by utilizing monocular depth and mesh reconstruction, thereby creating training data without requiring ground-truth depth. Besides, monocular depth estimation suffers from inherent scale ambiguity in real-world settings. To address this, in the second stage, we employ a scale- and shift-invariant loss (SSI Loss) to learn real-world scales when fine-tuning on real-world datasets. Our two-stage distillation framework enables depth completion models to harness the strengths of monocular foundation models. Experimental results demonstrate that models trained with our two-stage distillation framework achieve state-of-the-art performance, ranking \textbf{first place} on the KITTI benchmark. Code is available at https://github.com/Sharpiless/DMD3C
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。