arXiv:2506.21091cs.CV2025-06

用轻量结构提升小体积代价图的精度,实现实时高精度立体匹配。

ESMStereo: Enhanced ShuffleMixer Disparity Upsampling for Real-Time and Accurate Stereo Matching

  • 通过增强型交错混洗模块,将初始视差特征融合进上采样过程
  • 在高阶GPU上达116帧/秒,AGX Orin上达91帧/秒,保持高精度
  • 适合对速度与精度均有要求的自动驾驶等实时系统

立体匹配已成为现代自动驾驶系统的重要组成部分。开发兼具高精度与实时性的深度学习立体匹配模型仍是计算机视觉中的重大挑战。在基于代价图的立体匹配中,精确视差估计依赖于大规模代价图。然而,大体积代价图包含大量冗余信息,且需计算密集型聚合单元处理与回归,难以实现实时性能。相反,小体积代价图搭配轻量聚合单元虽能保障实时性,却因信息不足导致精度下降。为此,本文提出增强型交错混洗模块(ESM),缓解小体积代价图的信息损失。该模块将原始特征融入视差上采样单元,快速提取初始视差特征,并与图像特征融合。通过特征打乱与层分裂实现混合,再经紧凑的特征引导哑铃网络细化,恢复更精细的场景几何结构。ESM聚焦局部上下文连通性,具有大感受野与低计算开销,从而在实时条件下重建高精度视差图。ESMStereo紧凑版在高端GPU上达到116 FPS,在AGX Orin上达91 FPS。

原文摘要 · Abstract (English)

Stereo matching has become an increasingly important component of modern autonomous systems. Developing deep learning-based stereo matching models that deliver high accuracy while operating in real-time continues to be a major challenge in computer vision. In the domain of cost-volume-based stereo matching, accurate disparity estimation depends heavily on large-scale cost volumes. However, such large volumes store substantial redundant information and also require computationally intensive aggregation units for processing and regression, making real-time performance unattainable. Conversely, small-scale cost volumes followed by lightweight aggregation units provide a promising route for real-time performance, but lack sufficient information to ensure highly accurate disparity estimation. To address this challenge, we propose the Enhanced Shuffle Mixer (ESM) to mitigate information loss associated with small-scale cost volumes. ESM restores critical details by integrating primary features into the disparity upsampling unit. It quickly extracts features from the initial disparity estimation and fuses them with image features. These features are mixed by shuffling and layer splitting then refined through a compact feature-guided hourglass network to recover more detailed scene geometry. The ESM focuses on local contextual connectivity with a large receptive field and low computational cost, leading to the reconstruction of a highly accurate disparity map at real-time. The compact version of ESMStereo achieves an inference speed of 116 FPS on high-end GPUs and 91 FPS on the AGX Orin.

立体匹配实时推理轻量化模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。