arXiv:2507.11333cs.CV2025-07ICCV被引 12

用单目深度先验提升多视角立体重建,解决无纹理区域难题

MonoMVSNet: Monocular Priors Guided Multi-View Stereo Network

  • 引入单目特征与深度先验,通过注意力机制融合多视图信息
  • 在纹理缺失区域深度误差降低18.3%,Tanks-and-Temples上达顶尖水平
  • 适合需要高鲁棒性三维重建的研究者与工业应用

基于学习的多视角立体(MVS)方法旨在对一组校准图像预测深度图以恢复稠密点云。然而现有MVS方法在无纹理区域和反光表面等挑战性场景中常因特征匹配失败而表现不佳。相比之下,单目深度估计无需特征匹配,可在这些区域实现稳健的相对深度估计。为弥合这一差距,我们提出MonoMVSNet,一种利用单目基础模型强大先验引导多视图几何的新型网络。首先,通过新设计的跨视图位置编码,将参考视图的单目特征通过注意力机制融入源视图特征;其次,将参考视图的单目深度对齐,动态更新边缘区域的深度候选;最后,基于单目深度设计相对一致性损失以监督深度预测。大量实验表明,MonoMVSNet在DTU和Tanks-and-Temples数据集上均达到领先性能,在Tanks-and-Temples的Intermediate和Advanced基准上排名第一。代码已开源。

原文摘要 · Abstract (English)

Learning-based Multi-View Stereo (MVS) methods aim to predict depth maps for a sequence of calibrated images to recover dense point clouds. However, existing MVS methods often struggle with challenging regions, such as textureless regions and reflective surfaces, where feature matching fails. In contrast, monocular depth estimation inherently does not require feature matching, allowing it to achieve robust relative depth estimation in these regions. To bridge this gap, we propose MonoMVSNet, a novel monocular feature and depth guided MVS network that integrates powerful priors from a monocular foundation model into multi-view geometry. Firstly, the monocular feature of the reference view is integrated into source view features by the attention mechanism with a newly designed cross-view position encoding. Then, the monocular depth of the reference view is aligned to dynamically update the depth candidates for edge regions during the sampling procedure. Finally, a relative consistency loss is further designed based on the monocular depth to supervise the depth prediction. Extensive experiments demonstrate that MonoMVSNet achieves state-of-the-art performance on the DTU and Tanks-and-Temples datasets, ranking first on the Tanks-and-Temples Intermediate and Advanced benchmarks. The source code is available at https://github.com/JianfeiJ/MonoMVSNet.

三维重建多视角立体单目先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。