用扩散模型实现零样本单目深度补全，效果远超传统方法
Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
- 将深度补全重构为图像条件下的深度图生成任务
- 在极端稀疏深度观测下仍能生成高质量稠密深度图
- 适合跨场景、低密度深度数据的实时应用
深度补全将稀疏深度测量值升级为稠密深度图，依赖于常规图像作为引导。现有方法在严格受限环境下运行，当应用于训练域外图像或面对稀疏、不规则分布、密度多变的深度测量时表现不佳。受单目深度估计最新进展启发，我们将深度补全重新构想为基于稀疏测量的图像条件深度图生成。所提方法Marigold-DC基于预训练的单目深度估计潜在扩散模型，通过与去噪扩散迭代推理并行的优化方案，在测试时注入深度观测作为引导。该方法在多种环境间展现出卓越的零样本泛化能力，可有效处理极稀疏引导数据。结果表明，当代单目深度先验显著增强了深度补全的鲁棒性：与其视作以图像为引导的稀疏深度插值，不如看作从（稠密）图像像素中恢复稠密深度，由稀疏深度进行指导。
原文摘要 · Abstract (English)
Depth completion upgrades sparse depth measurements into dense depth maps guided by a conventional image. Existing methods for this highly ill-posed task operate in tightly constrained settings and tend to struggle when applied to images outside the training domain or when the available depth measurements are sparse, irregularly distributed, or of varying density. Inspired by recent advances in monocular depth estimation, we reframe depth completion as an image-conditional depth map generation guided by sparse measurements. Our method, Marigold-DC, builds on a pretrained latent diffusion model for monocular depth estimation and injects the depth observations as test-time guidance via an optimization scheme that runs in tandem with the iterative inference of denoising diffusion. The method exhibits excellent zero-shot generalization across a diverse range of environments and handles even extremely sparse guidance effectively. Our results suggest that contemporary monocular depth priors greatly robustify depth completion: it may be better to view the task as recovering dense depth from (dense) image pixels, guided by sparse depth; rather than as inpainting (sparse) depth, guided by an image. Project website: https://MarigoldDepthCompletion.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。