用SfM信息提升单目深度估计,直接用于多视角重建
Multi-view Reconstruction via SfM-guided Monocular Depth Estimation
- 将SfM多视角先验融入单目深度估计流程
- 在真实场景数据上显著提升深度预测精度
- 适用于室内、街景、航拍等多种场景重建
本文提出一种新的多视角几何重建方法。近年来,大型视觉模型发展迅速,在各类任务中表现优异,展现出强大泛化能力。已有研究利用大型视觉模型进行单目深度估计,间接辅助多视角重建。然而,由于单目深度估计固有的模糊性,预测深度通常不够精确,限制了其在多视角重建中的应用。本文提出将SfM提供的强多视角先验信息引入深度估计过程,从而提升深度预测质量,并实现其在多视角几何重建中的直接应用。在多个公开真实场景数据集上的实验结果表明,本方法相比以往单目深度估计方法显著提升了深度估计质量。此外,我们在包括室内、街景和航拍在内的多种场景类型中评估了重建效果,优于当前最先进的多视图立体(MVS)方法。代码与补充材料见 https://zju3dv.github.io/murre/。
原文摘要 · Abstract (English)
In this paper, we present a new method for multi-view geometric reconstruction. In recent years, large vision models have rapidly developed, performing excellently across various tasks and demonstrating remarkable generalization capabilities. Some works use large vision models for monocular depth estimation, which have been applied to facilitate multi-view reconstruction tasks in an indirect manner. Due to the ambiguity of the monocular depth estimation task, the estimated depth values are usually not accurate enough, limiting their utility in aiding multi-view reconstruction. We propose to incorporate SfM information, a strong multi-view prior, into the depth estimation process, thus enhancing the quality of depth prediction and enabling their direct application in multi-view geometric reconstruction. Experimental results on public real-world datasets show that our method significantly improves the quality of depth estimation compared to previous monocular depth estimation works. Additionally, we evaluate the reconstruction quality of our approach in various types of scenes including indoor, streetscape, and aerial views, surpassing state-of-the-art MVS methods. The code and supplementary materials are available at https://zju3dv.github.io/murre/ .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。