arXiv:2508.04984cs.CV2025-08中稿 · IEEE TIP被引 3

用深度基础模型提升稀疏深度补全的泛化能力,无需大量训练数据。

Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completion

  • 利用深度基础模型提取图像中的结构与语义线索,指导深度信息传播。
  • 无参数双空间传播保持几何结构与局部一致性,显著提升外分布场景表现。
  • 适合追求高鲁棒性、少训练数据的深度补全应用,如自动驾驶感知系统。

深度补全是计算机视觉的关键挑战,旨在从稀疏深度图和配对的RGB图像中重建稠密深度图。现有学习方法依赖精心准备但有限的数据,在分布外(OOD)场景下性能显著下降。近期的基础模型通过大规模训练展现出卓越的单目深度估计鲁棒性,将其用于增强深度补全模型的鲁棒性是一条有前景的路径。本文提出一种新型深度补全框架,借助深度基础模型实现无需大规模训练的优异鲁棒性。具体而言,我们利用深度基础模型从RGB图像中提取环境线索(包括结构与语义上下文),以引导稀疏深度信息在缺失区域的传播。进一步设计了无可学习参数的双空间传播机制,在3D与2D空间中协同传播,有效维持几何结构与局部一致性。为精细化调整复杂结构,引入可学习修正模块,逐步优化深度预测结果。我们在NYUv2和KITTI数据集上进行内分布训练,并在16个其他数据集上广泛评估。实验表明,该框架在各类外分布场景下表现优异,超越现有最优深度补全方法。模型代码已开源:https://github.com/shenglunch/PSD。

原文摘要 · Abstract (English)

Depth completion is a pivotal challenge in computer vision, aiming at reconstructing the dense depth map from a sparse one, typically with a paired RGB image. Existing learning based models rely on carefully prepared but limited data, leading to significant performance degradation in out-of-distribution (OOD) scenarios. Recent foundation models have demonstrated exceptional robustness in monocular depth estimation through large-scale training, and using such models to enhance the robustness of depth completion models is a promising solution. In this work, we propose a novel depth completion framework that leverages depth foundation models to attain remarkable robustness without large-scale training. Specifically, we leverage a depth foundation model to extract environmental cues, including structural and semantic context, from RGB images to guide the propagation of sparse depth information into missing regions. We further design a dual-space propagation approach, without any learnable parameters, to effectively propagates sparse depth in both 3D and 2D spaces to maintain geometric structure and local consistency. To refine the intricate structure, we introduce a learnable correction module to progressively adjust the depth prediction towards the real depth. We train our model on the NYUv2 and KITTI datasets as in-distribution datasets and extensively evaluate the framework on 16 other datasets. Our framework performs remarkably well in the OOD scenarios and outperforms existing state-of-the-art depth completion methods. Our models are released in https://github.com/shenglunch/PSD.

深度补全基础模型泛化能力稀疏深度

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。