arXiv:2409.00638cs.CV2024-09TPAMI被引 109

提出新架构,高效解决大视差与模糊区域的立体匹配难题

IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching

论文配图:IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching
图 1 · 摘自论文原文
  • 构建多尺度几何编码体积,兼顾大视差与细节信息
  • 自适应匹配模块提升大视差区域计算效率,误差率降低超50%
  • 适合实时立体视觉系统,尤其在纹理缺失与遮挡场景表现优异

立体匹配是计算机视觉与机器人系统的核心组件。尽管过去十年取得显著进展,但在病态区域和大视差下的匹配歧义仍是开放挑战。本文提出一种新型深度网络架构 IGEV++,构建多范围几何编码体积(MGEV),对病态区域和大视差编码粗粒度几何信息,同时保留小视差与细节的细粒度信息。通过自适应补丁匹配模块,高效计算大视差范围及病态区域的匹配代价;进一步设计选择性几何特征融合模块,自适应融合多范围、多粒度的几何特征。随后将融合特征输入 ConvGRU 进行迭代更新,实现快速收敛。IGEV++ 在 Scene Flow 测试集上对所有视差范围(最高达768像素)均取得最佳性能,并在 Middlebury、ETH3D、KITTI 2012 和 2015 基准测试中达到当前最优精度。特别地,在大视差基准 Middlebury 上,2像素误差点率(Bad 2.0)仅为3.23%,相比 RAFT-Stereo 和 GMStereo 分别降低31.9%和54.8%。此外,还提出了实时版本,在 KITTI 基准上优于所有已发表的实时方法。代码已开源。

原文摘要 · Abstract (English)

Stereo matching is a core component in many computer vision and robotics systems. Despite significant advances over the last decade, handling matching ambiguities in ill-posed regions and large disparities remains an open challenge. In this paper, we propose a new deep network architecture, called IGEV++, for stereo matching. The proposed IGEV++ constructs Multi-range Geometry Encoding Volumes (MGEV), which encode coarse-grained geometry information for ill-posed regions and large disparities, while preserving fine-grained geometry information for details and small disparities. To construct MGEV, we introduce an adaptive patch matching module that efficiently and effectively computes matching costs for large disparity ranges and/or ill-posed regions. We further propose a selective geometry feature fusion module to adaptively fuse multi-range and multi-granularity geometry features in MGEV. Then, we input the fused geometry features into ConvGRUs to iteratively update the disparity map. MGEV allows to efficiently handle large disparities and ill-posed regions, such as occlusions and textureless regions, and enjoys rapid convergence during iterations. Our IGEV++ achieves the best performance on the Scene Flow test set across all disparity ranges, up to 768px. Our IGEV++ also achieves state-of-the-art accuracy on the Middlebury, ETH3D, KITTI 2012, and 2015 benchmarks. Specifically, IGEV++ achieves a 3.23\% 2-pixel outlier rate (Bad 2.0) on the large disparity benchmark, Middlebury, representing error reductions of 31.9\% and 54.8\% compared to RAFT-Stereo and GMStereo, respectively. We also present a real-time version of IGEV++ that achieves the best performance among all published real-time methods on the KITTI benchmarks. The code is publicly available at https://github.com/gangweix/IGEV and https://github.com/gangweix/IGEV-plusplus.

立体匹配几何编码大视差实时系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。