提出一种能零样本适配多种场景的鲁棒深度补全方法
OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration
- 设计多分辨率深度融合模块，处理极稀疏输入
- 在7个数据集上误差降低最高达43%
- 适合需要跨域泛化的深度感知应用
深度补全（DC）旨在从彩色图像和稀疏深度图中预测稠密深度图。现有方法在新数据集或未见稀疏模式下泛化能力差，限制了实际应用。本文提出OMNI-DC，一种具有强零样本泛化能力的深度补全模型。核心是新型多分辨率深度整合器，可有效处理极稀疏深度输入。同时引入拉普拉斯损失以建模训练过程中的不确定性。模型在高质量数据集混合数据上训练，并采用尺度归一化技术及合成深度模式。在7个数据集上的大量实验表明，相比基线方法均有显著提升，误差最高降低43%。代码与模型权重已开源。
原文摘要 · Abstract (English)
Depth completion (DC) aims to predict a dense depth map from an RGB image and a sparse depth map. Existing DC methods generalize poorly to new datasets or unseen sparse depth patterns, limiting their real-world applications. We propose OMNI-DC, a highly robust DC model that generalizes well zero-shot to various datasets. The key design is a novel Multi-resolution Depth Integrator, allowing our model to deal with very sparse depth inputs. We also introduce a novel Laplacian loss to model the ambiguity in the training process. Moreover, we train OMNI-DC on a mixture of high-quality datasets with a scale normalization technique and synthetic depth patterns. Extensive experiments on 7 datasets show consistent improvements over baselines, reducing errors by as much as 43%. Codes and checkpoints are available at https://github.com/princeton-vl/OMNI-DC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。