arXiv:2412.14103cs.CV2024-12被引 7

无需微调,用传感器点云重缩放即可获得精确单目深度估计。

A Simple yet Effective Test-Time Adaptation for Zero-Shot Monocular Metric Depth Estimation

  • 利用激光雷达或运动恢复结构的3D点云对深度图进行无监督重缩放。
  • 在多个基准上提升精度,媲美微调方法且更鲁棒。
  • 适合部署受限、无法收集标注数据的实时系统应用。

近年来,基于基础模型的单目深度估计(如Depth Anything)推动了零样本单目度量深度估计的发展。由于其输出为仿射不变的视差图,通常需通过微调模型来恢复度量深度,但该过程耗时耗力,且依赖目标相机采集的数据集及真实值标注,同时可能损害原始模型的泛化能力。本文提出一种新方法,仅利用低分辨率激光雷达或结合惯性测量单元(IMU)位姿的运动恢复结构获取的3D点云,对Depth Anything的预测结果进行重缩放,避免微调,保持模型泛化能力,并对稀疏深度噪声、相机-激光雷达标定误差和深度模型噪声具有强鲁棒性。实验表明,该方法在零样本单目度量深度估计任务中显著优于现有方法,性能媲美微调方案,且优于深度补全类方法。代码已开源:github.com/ENSTA-U2IS-AI/depth-rescaling。

原文摘要 · Abstract (English)

The recent development of \emph{foundation models} for monocular depth estimation such as Depth Anything paved the way to zero-shot monocular depth estimation. Since it returns an affine-invariant disparity map, the favored technique to recover the metric depth consists in fine-tuning the model. However, this stage is not straightforward, it can be costly and time-consuming because of the training and the creation of the dataset. The latter must contain images captured by the camera that will be used at test time and the corresponding ground truth. Moreover, the fine-tuning may also degrade the generalizing capacity of the original model. Instead, we propose in this paper a new method to rescale Depth Anything predictions using 3D points provided by sensors or techniques such as low-resolution LiDAR or structure-from-motion with poses given by an IMU. This approach avoids fine-tuning and preserves the generalizing power of the original depth estimation model while being robust to the noise of the sparse depth, of the camera-LiDAR calibration or of the depth model. Our experiments highlight enhancements relative to zero-shot monocular metric depth estimation methods, competitive results compared to fine-tuned approaches and a better robustness than depth completion approaches. Code available at github.com/ENSTA-U2IS-AI/depth-rescaling.

深度估计测试时自适应零样本传感器融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。