用相机位姿准确率评估单目深度估计,更真实反映实际应用效果。
Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth

- 以位姿估计精度为指标,替代传统深度误差衡量方法。
- 在复杂场景下,部分高精度深度模型泛化能力不足。
- 仅需相机位姿即可评估,适合无真值深度的大规模场景。
近年来,单目深度估计因模型能力和大规模训练数据的进步显著提升,其预测深度常被用于SfM、视觉定位和SLAM等下游任务。然而,现有评估仍主要依赖深度精度指标,这些全局误差聚合指标未必能反映深度对几何任务的实际帮助。为此,本文提出Depth2Pose,一种基于下游任务的单目深度估计算法评估框架。通过将深度预测与深度感知的特征匹配结合,在几何求解器中以相对相机位姿估计精度作为深度质量的任务驱动代理指标。传统基准需要每像素深度真值,获取成本高;而本方法仅需相机位姿,可通过SfM高效估计。因此可应用于真值深度难以获取的场景,如大尺度或严重遮挡环境(如植被覆盖区域)。基于此,我们构建了D2P数据集,包含分布外的挑战性场景。实验表明,虽在标准基准上表现优异的方法在相同数据集上也具备良好位姿估计性能,但未必能在我们的新数据集上泛化。最后,我们提供了一个简单且可扩展的评估框架。代码与数据集已公开于kocurvik.github.io/depth2pose。
原文摘要 · Abstract (English)
Monocular depth estimation has improved significantly in recent years, driven by increasingly powerful models and large-scale training data. Predicted depth is increasingly used as an input signal for downstream tasks such as Structure-from-Motion (SfM), visual localization, and SLAM. However, monocular depth estimators (MDEs) are still primarily evaluated in terms of depth accuracy. Standard metrics aggregate errors globally and may not reflect the usefulness of depth for downstream geometric tasks. We therefore propose Depth2Pose, a framework for evaluating MDEs in the context of downstream tasks. By combining depth predictions with feature correspondences in depth-aware geometric solvers, we use relative camera pose estimation accuracy as a task-driven proxy for depth quality. Traditional benchmarks require dense ground truth in the form of per-pixel depth, which is expensive to obtain. In contrast, our formulation requires only camera poses, which can be estimated efficiently, e.g., using Structure-from-Motion pipelines. As a result, our framework can be applied to scenes where ground-truth depth is difficult to obtain, for example due to large scene scale or heavy occlusions (e.g., vegetated environments). Leveraging this, we introduce the D2P dataset, which contains challenging scenes outside the distribution of commonly used training data. We show that methods performing well under standard depth error metrics on existing benchmarks also perform well under our pose-based metric when evaluated on the same datasets, but do not necessarily generalize to our more challenging dataset. Finally, we provide a simple and extensible evaluation framework. The dataset and code are available at kocurvik.github.io/depth2pose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。