统一处理广角与全景图的单目深度估计,效果超越专攻模型。
DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images

- 将全景图拆解为重叠的透视块,用一致性损失融合。
- 仅用一个全景数据集训练,零样本跨13数据集领先。
- 无需特殊网络结构,兼容标准Transformer,适合多场景部署。
尽管单目深度估计已取得显著进展,但实现窄视场(FoV)透视图与360°全景图的通用度量深度估计仍是未解难题。现有方法通常针对特定相机类型设计,难以在不同场景间泛化。根本原因在于透视与全景相机之间的几何差异,以及带度量标注的全景数据稀缺。本文提出DepthMaster,一种统一的度量深度估计框架。不采用专门网络学习球面畸变,而是将全景图分解为重叠的透视块。关键在于,不同于以往依赖人工架构调整处理边界的方法,我们引入新的对应一致性损失(CCL),并注入虚拟投影相机作为几何先验,实现无缝拼接,避免专用算子,同时保持主干网络与标准Transformer高度兼容。该策略通过统一输入至标准透视表示,解决几何差异问题,并通过直接利用大量透视数据集中的强大度量先验,缓解数据稀缺。仅使用一个全景数据集训练,DepthMaster在13个多样数据集上实现零样本最优表现,不仅优于通用方法,更超越了视角与全景领域的顶尖专用模型。
原文摘要 · Abstract (English)
While monocular depth estimation has achieved significant progress, achieving generalized metric depth estimation for both narrow field-of-view (FoV) perspectives and $360^\circ$ panoramas remains an unsolved challenge. Existing methods are often tailored to specific camera types and struggle to produce accurate metric depth that generalizes across diverse settings. This limitation stems from two key challenges: the inherent geometric discrepancy between perspective and panoramic cameras, and the scarcity of panoramic training data with metric annotations. In this work, we introduce DepthMaster, a unified metric depth estimation framework. Rather than employing specialized networks to learn spherical distortions, we reformulate the problem by decomposing panoramic images into overlapping perspective patches. Crucially, distinct from prior projection-based methods that rely on ad-hoc architectural modifications to handle boundaries, we introduce a novel Correspondence Consistency Loss (CCL) and inject virtual projection cameras as geometric priors, allowing us to seamlessly stitch the patches while avoiding specialized operators and keeping the backbone largely compatible with standard Transformer designs. This strategy also resolves the geometric differences by unifying all inputs into a canonical perspective representation, and effectively circumvents data scarcity by directly unlocking powerful metric priors from vast perspective datasets. Trained on a mixed dataset that contains only one panorama dataset, DepthMaster achieves state-of-the-art zero-shot performance on 13 diverse datasets, outperforming not only universal methods but also leading specialist models in both perspective and panoramic domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。