arXiv:2505.03470cs.CVcs.AI2025-05中稿 · Neurocomputing被引 2

让深度学习在训练时直接检查多视角几何一致性,提升3D重建速度与精度。

Blending 3D Geometry and Machine Learning for Multi-View Stereopsis

  • 训练中实时校验多视角、多尺度几何一致性,而非仅事后处理
  • 训练迭代次数减少一半,DTU和BlendedMVS上达到新SOTA
  • 适合追求高精度3D重建的计算机视觉研究者与工程师

传统多视图立体(MVS)方法依赖光度与几何一致性约束。现代基于学习的方法通常使用平面扫描算法推断3D结构,仅在后处理阶段施加显式的几何一致性(GC)检查,且不影响学习过程。本文提出GC MVSNet plus plus,首次在学习阶段主动强制参考视图深度图在多个源视图及不同尺度下的几何一致性。该集成式GC检查通过直接惩罚几何不一致像素,显著加速训练,使训练迭代次数减半。此外,设计了两种块结构的密集连接代价正则化网络:简单型与特征密集型,以增强密集特征连接带来的正则化效果。大量实验表明,本方法在DTU和BlendedMVS数据集上达到新SOTA,Tanks and Temples基准中位列第二。据我们所知,这是首个在学习过程中实现多视图、多尺度监督几何一致性的方法。代码已开源。

原文摘要 · Abstract (English)

Traditional multi-view stereo (MVS) methods primarily depend on photometric and geometric consistency constraints. In contrast, modern learning-based algorithms often rely on the plane sweep algorithm to infer 3D geometry, applying explicit geometric consistency (GC) checks only as a post-processing step, with no impact on the learning process itself. In this work, we introduce GC MVSNet plus plus, a novel approach that actively enforces geometric consistency of reference view depth maps across multiple source views (multi view) and at various scales (multi scale) during the learning phase (see Fig. 1). This integrated GC check significantly accelerates the learning process by directly penalizing geometrically inconsistent pixels, effectively halving the number of training iterations compared to other MVS methods. Furthermore, we introduce a densely connected cost regularization network with two distinct block designs simple and feature dense optimized to harness dense feature connections for enhanced regularization. Extensive experiments demonstrate that our approach achieves a new state of the art on the DTU and BlendedMVS datasets and secures second place on the Tanks and Temples benchmark. To our knowledge, GC MVSNet plus plus is the first method to enforce multi-view, multi-scale supervised geometric consistency during learning. Our code is available.

3D重建多视图立体深度学习几何一致性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。