用2D卷积实现移动端立体匹配，清晰保留边缘和细节
BANet: Bilateral Aggregation Network for Mobile Stereo Matching
- 分离成本体为细节与平滑部分，分别聚合后融合
- KITTI 2015上比MobileStereoNet-2D高35.3%准确率，运行更快
- 仅用2D卷积，适合手机部署，避免复杂操作
当前先进的立体匹配方法通常使用计算开销大的3D卷积来聚合完整成本体，但其高计算需求使得在移动设备上部署困难。直接使用2D卷积进行成本聚合常导致边缘模糊、细节丢失以及无纹理区域的误匹配。虽然可变形卷积和迭代光流等复杂操作可部分缓解此问题，但它们并不适合移动端，限制了在移动设备上的应用。本文提出一种新型的双边聚合网络（BANet），仅用2D卷积即可实现高质量的移动端立体匹配，生成具有锐利边缘和精细细节的结果。具体而言，我们首先利用空间注意力图将完整成本体分为细节型和光滑型两部分，随后分别进行细节与光滑聚合，最终融合两者得到最终视差图。实验结果表明，我们的BANet-2D显著优于其他移动端友好的方法，在KITTI 2015排行榜上比MobileStereoNet-2D高出35.3%的准确率，同时在移动设备上运行更快。代码已开源：https://github.com/gangweix/BANet。
原文摘要 · Abstract (English)
State-of-the-art stereo matching methods typically use costly 3D convolutions to aggregate a full cost volume, but their computational demands make mobile deployment challenging. Directly applying 2D convolutions for cost aggregation often results in edge blurring, detail loss, and mismatches in textureless regions. Some complex operations, like deformable convolutions and iterative warping, can partially alleviate this issue; however, they are not mobile-friendly, limiting their deployment on mobile devices. In this paper, we present a novel bilateral aggregation network (BANet) for mobile stereo matching that produces high-quality results with sharp edges and fine details using only 2D convolutions. Specifically, we first separate the full cost volume into detailed and smooth volumes using a spatial attention map, then perform detailed and smooth aggregations accordingly, ultimately fusing both to obtain the final disparity map. Experimental results demonstrate that our BANet-2D significantly outperforms other mobile-friendly methods, achieving 35.3\% higher accuracy on the KITTI 2015 leaderboard than MobileStereoNet-2D, with faster runtime on mobile devices. Code: \textcolor{magenta}{https://github.com/gangweix/BANet}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。