用扩散模型修复缺失深度图，能准确补全连续区域和孤立点。
DepthLab: From Partial to Complete
- 基于图像扩散先验，从部分深度图生成完整深度图。
- 在多个任务中优于现有方法，视觉质量与数值指标双提升。
- 适合需要高质量深度补全的3D重建与生成任务。
深度数据中的缺失值在诸多应用中仍是常见挑战，源于数据采集不全或视角变化等。本文提出DepthLab，一种基于图像扩散先验的深度补全基础模型。该模型具备两大优势：（1）对深度缺失区域具有强鲁棒性，可有效补全连续区域与孤立点；（2）在填充时能忠实保持与已知深度一致的尺度一致性。基于此，该方法在多项下游任务中表现卓越，包括3D场景补全、文本到3D场景生成、基于DUST3R的稀疏视图重建以及LiDAR深度补全，均在数值性能与视觉质量上超越现有方案。项目主页及源码见 https://johanan528.github.io/depthlab_web/。
原文摘要 · Abstract (English)
Missing values remain a common challenge for depth data across its wide range of applications, stemming from various causes like incomplete data acquisition and perspective alteration. This work bridges this gap with DepthLab, a foundation depth inpainting model powered by image diffusion priors. Our model features two notable strengths: (1) it demonstrates resilience to depth-deficient regions, providing reliable completion for both continuous areas and isolated points, and (2) it faithfully preserves scale consistency with the conditioned known depth when filling in missing values. Drawing on these advantages, our approach proves its worth in various downstream tasks, including 3D scene inpainting, text-to-3D scene generation, sparse-view reconstruction with DUST3R, and LiDAR depth completion, exceeding current solutions in both numerical performance and visual quality. Our project page with source code is available at https://johanan528.github.io/depthlab_web/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。