通过融合深度边缘与可见性先验,提升多视角立体重建的鲁棒性。
DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo
- 利用深度边缘对齐与跨视图可见性先验指导补丁变形。
- 在ETH3D和Tanks & Temples上达到当前最优性能。
- 适合需要高精度三维重建的研究者和工业应用。
基于补丁变形的方法在多视角立体重建中表现优异,因其能通过可变形感知恢复无纹理区域。然而,现有方法通常仅关注相关可靠像素以缓解匹配模糊,却忽略了因误跳边缘和可见性遮挡导致的变形不稳定性,易引发估计偏差。为此,本文提出DVP-MVS,创新性地融合深度-边缘对齐与跨视图先验,实现鲁棒且感知可见性的补丁变形。首先,采用Depth Anything V2结合Roberts算子分别初始化粗略深度图与边缘图,并通过膨胀-腐蚀策略对齐生成细粒度均匀边界,用于引导补丁变形。其次,将视图选择权重重构为可见性图,通过跨视图深度重投影恢复可见区域,作为跨视图先验,增强可见性感知的变形能力。最后,引入聚合可见半球法向量(基于视图选择)与基于极线的局部投影深度差异,分别提升传播与精化过程中的多视图几何一致性。在ETH3D与Tanks & Temples基准上的大量实验表明,本方法在鲁棒性与泛化性方面均达到当前最优水平。
原文摘要 · Abstract (English)
Patch deformation-based methods have recently exhibited substantial effectiveness in multi-view stereo, due to the incorporation of deformable and expandable perception to reconstruct textureless areas. However, such approaches typically focus on exploring correlative reliable pixels to alleviate match ambiguity during patch deformation, but ignore the deformation instability caused by mistaken edge-skipping and visibility occlusion, leading to potential estimation deviation. To remedy the above issues, we propose DVP-MVS, which innovatively synergizes depth-edge aligned and cross-view prior for robust and visibility-aware patch deformation. Specifically, to avoid unexpected edge-skipping, we first utilize Depth Anything V2 followed by the Roberts operator to initialize coarse depth and edge maps respectively, both of which are further aligned through an erosion-dilation strategy to generate fine-grained homogeneous boundaries for guiding patch deformation. In addition, we reform view selection weights as visibility maps and restore visible areas by cross-view depth reprojection, then regard them as cross-view prior to facilitate visibility-aware patch deformation. Finally, we improve propagation and refinement with multi-view geometry consistency by introducing aggregated visible hemispherical normals based on view selection and local projection depth differences based on epipolar lines, respectively. Extensive evaluations on ETH3D and Tanks & Temples benchmarks demonstrate that our method can achieve state-of-the-art performance with excellent robustness and generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。