用稀疏深度点引导扩散模型，修复严重缺失的深度图
SteeredMarigold: Steering Diffusion Towards Depth Completion of Largely Incomplete Depth Maps
- 用已有稀疏深度点作为条件，控制扩散模型生成完整深度图
- 在NYUv2数据集上，大范围无深度区域下仍达领先性能
- 无需训练、零样本适配，适合真实场景中深度缺失严重的应用
尽管实际环境中部署的RGB-D传感器获取的深度图常存在大面积无效深度值，但多数深度补全方法仍假设场景中所有区域都有深度值。为解决此问题，我们提出SteeredMarigold，一种无需训练、零样本的深度补全方法，可生成度量级稠密深度图，即使面对严重不完整的深度图亦有效。该方法利用可用的稀疏深度点作为条件，引导去噪扩散概率模型生成完整深度。在NYUv2数据集测试中，当大范围区域无深度输入时，其性能优于现有顶尖方法，展现出对深度图不完整性极强的鲁棒性。代码已公开于https://steeredmarigold.github.io。
原文摘要 · Abstract (English)
Even if the depth maps captured by RGB-D sensors deployed in real environments are often characterized by large areas missing valid depth measurements, the vast majority of depth completion methods still assumes depth values covering all areas of the scene. To address this limitation, we introduce SteeredMarigold, a training-free, zero-shot depth completion method capable of producing metric dense depth, even for largely incomplete depth maps. SteeredMarigold achieves this by using the available sparse depth points as conditions to steer a denoising diffusion probabilistic model. Our method outperforms relevant top-performing methods on the NYUv2 dataset, in tests where no depth was provided for a large area, achieving state-of-art performance and exhibiting remarkable robustness against depth map incompleteness. Our source code is publicly available at https://steeredmarigold.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。