融合深度法向边缘与可视性先验,提升多视角立体重建的稳定性与精度。
DVP-MVS++: Synergize Depth-Normal-Edge and Harmonized Visibility Prior for Multi-View Stereo
- 通过边缘对齐与可见性加权,增强Patch变形的鲁棒性。
- 在ETH3D等数据集上达到最新最优性能,尤其在纹理缺失区域表现优异。
- 适合关注高精度三维重建与视觉一致性优化的研究者。
近期基于块形变的方法在多视角立体重建中表现出色,因其能灵活感知无纹理区域。然而,这类方法通常仅关注可靠像素匹配以缓解形变歧义,忽视了因边缘遗漏和可见性遮挡导致的形变不稳定性,可能引发估计偏差。为此,本文提出DVP-MVS++,通过协同融合深度-法向-边缘对齐与协调一致的跨视图先验,实现鲁棒且感知可视性的块形变。首先,利用DepthPro、Metric3Dv2和Roberts算子分别生成粗略深度图、法向图与边缘图,并通过膨胀-腐蚀策略对齐,形成精细均匀边界以支持稳定形变。其次,将视图选择权重重构为可见性图,结合增强的跨视图深度重投影与面积最大化策略,可靠恢复可见区域并有效平衡形变块,从而获得协调一致的跨视图先验。此外,通过聚合选定视角的法向与基线方向投影深度差,建立几何一致性,并采用SHIQ进行高光修正,实现高光感知的一致性,显著提升传播与精化阶段的重建质量。在ETH3D、Tanks & Temples和Strecha数据集上的实验表明,该方法性能达当前最优,具备强泛化能力。
原文摘要 · Abstract (English)
Recently, patch deformation-based methods have demonstrated significant effectiveness in multi-view stereo due to their incorporation of deformable and expandable perception for reconstructing textureless areas. However, these methods generally focus on identifying reliable pixel correlations to mitigate matching ambiguity of patch deformation, while neglecting the deformation instability caused by edge-skipping and visibility occlusions, which may cause potential estimation deviations. To address these issues, we propose DVP-MVS++, an innovative approach that synergizes both depth-normal-edge aligned and harmonized cross-view priors for robust and visibility-aware patch deformation. Specifically, to avoid edge-skipping, we first apply DepthPro, Metric3Dv2 and Roberts operator to generate coarse depth maps, normal maps and edge maps, respectively. These maps are then aligned via an erosion-dilation strategy to produce fine-grained homogeneous boundaries for facilitating robust patch deformation. Moreover, we reformulate view selection weights as visibility maps, and then implement both an enhanced cross-view depth reprojection and an area-maximization strategy to help reliably restore visible areas and effectively balance deformed patch, thus acquiring harmonized cross-view priors for visibility-aware patch deformation. Additionally, we obtain geometry consistency by adopting both aggregated normals via view selection and projection depth differences via epipolar lines, and then employ SHIQ for highlight correction to enable geometry consistency with highlight-aware perception, thus improving reconstruction quality during propagation and refinement stage. Evaluation results on ETH3D, Tanks & Temples and Strecha datasets exhibit the state-of-the-art performance and robust generalization capability of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。