用2D卷积解耦4D代价图,实现高效实时立体匹配
Decoupling Bidirectional Geometric Representations of 4D cost volume with 2D convolution
- 将4D代价图的空间与视差信息分离处理,仅用2D卷积完成聚合
- 在保持实时性能的同时,精度超越现有基于聚合的方法
- 适合移动端部署,为立体匹配提供轻量级新范式
高性能实时立体匹配方法通常依赖3D代价体积正则化,但对移动设备不友好;而基于2D正则化的方法在病态区域表现不佳。本文提出一种部署友好的4D代价聚合网络DBStereo,完全基于2D卷积。首先系统分析了4D代价体积的解耦特性,并设计轻量级双向几何聚合模块,分别捕捉空间与视差表示。通过解耦学习,该方法在保持实时性的同时实现优异精度。大量实验表明,DBStereo在推理速度和准确性上均优于所有现有基于聚合的方法,甚至超过迭代式方法IGEV-Stereo。本研究打破使用3D卷积处理4D代价体积的惯常设计,为后续研究提供了简单而强大的解耦聚合基线。代码即将开源。
原文摘要 · Abstract (English)
High-performance real-time stereo matching methods invariably rely on 3D regularization of the cost volume, which is unfriendly to mobile devices. And 2D regularization based methods struggle in ill-posed regions. In this paper, we present a deployment-friendly 4D cost aggregation network DBStereo, which is based on pure 2D convolutions. Specifically, we first provide a thorough analysis of the decoupling characteristics of 4D cost volume. And design a lightweight bidirectional geometry aggregation block to capture spatial and disparity representation respectively. Through decoupled learning, our approach achieves real-time performance and impressive accuracy simultaneously. Extensive experiments demonstrate that our proposed DBStereo outperforms all existing aggregation-based methods in both inference time and accuracy, even surpassing the iterative-based method IGEV-Stereo. Our study break the empirical design of using 3D convolutions for 4D cost volume and provides a simple yet strong baseline of the proposed decouple aggregation paradigm for further study. Code will be available at (\href{https://github.com/happydummy/DBStereo}{https://github.com/happydummy/DBStereo}) soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。