无需深度范围先验的多视角立体重建方法，提升几何建模精度。
A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding
- 采用多视角视差注意力模块，统一融合所有源图像上下文信息。
- 在DTU和Tanks&Temple上实现优于现有方法的重建精度，尤其在复杂场景中表现突出。
- 适合需要高精度三维重建的应用，如数字孪生、机器人导航等。
本文提出一种新型多视角立体（MVS）框架，摆脱深度范围先验约束。与以往基于成对处理的无先验方法不同，本方法同时考虑所有源图像。我们引入多视角视差注意力（MDA）模块，聚合多视图图像内及跨图像的长程上下文信息。针对极线视差流的非对称性，关键在于准确建模多视角几何约束。通过引入位姿嵌入，整合相机位姿等信息，为以注意力主导的多视角视差特征融合提供隐式几何约束。此外，由于同一参考像素在不同源图像中观测质量差异显著，我们为每张源图像构建对应隐藏状态，并显式估计源图像极线采样点处当前像素的质量，通过不确定性估计模块动态更新隐藏状态。在DTU数据集和Tanks&Temple基准上的大量实验验证了方法的有效性。代码已公开于项目页面：https://zju3dv.github.io/GD-PoseMVS/。
原文摘要 · Abstract (English)
In this paper, we propose a novel multi-view stereo (MVS) framework that gets rid of the depth range prior. Unlike recent prior-free MVS methods that work in a pair-wise manner, our method simultaneously considers all the source images. Specifically, we introduce a Multi-view Disparity Attention (MDA) module to aggregate long-range context information within and across multi-view images. Considering the asymmetry of the epipolar disparity flow, the key to our method lies in accurately modeling multi-view geometric constraints. We integrate pose embedding to encapsulate information such as multi-view camera poses, providing implicit geometric constraints for multi-view disparity feature fusion dominated by attention. Additionally, we construct corresponding hidden states for each source image due to significant differences in the observation quality of the same pixel in the reference frame across multiple source frames. We explicitly estimate the quality of the current pixel corresponding to sampled points on the epipolar line of the source image and dynamically update hidden states through the uncertainty estimation module. Extensive results on the DTU dataset and Tanks&Temple benchmark demonstrate the effectiveness of our method. The code is available at our project page: https://zju3dv.github.io/GD-PoseMVS/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。