用更稳健的单目深度嵌入提升立体匹配泛化能力
Rethinking Monocular Depth Embedding for Generalized Stereo Matching

- 减少分支耦合,用软约束替代硬约束避免错误单目信息干扰
- 在特征提取和GRU迭代中融合单目深度,提升边界感知与精度
- 提出边缘置信度估计与感知损失,缓解数据增强导致的边界模糊
单目方法虽具丰富上下文先验但几何精度不足,立体匹配方法虽几何准确却在无纹理和遮挡区域表现差。现有方法尝试对齐单目深度与立体信息以增强泛化性,但稳定对齐困难,且不可靠的单目线索会显著降低性能。本文重新思考单目深度嵌入:首先,通过减少分支耦合而非扩展网络宽度来防止捷径学习;其次,构建来自单目深度的软约束以提高对误差的容忍度。基于此,将单目信息融入特征提取与GRU迭代过程。具体地,将单目深度图与RGB图像融合,增强深度边界感知并抑制匹配歧义;融合图像用于特征提取,使上下文特征编码全局几何信息。此外,利用单目深度梯度特征引导视差更新,帮助跳出局部震荡。最后,针对数据增强导致监督视差边界模糊的问题,提出边缘置信度估计与边缘感知损失函数。本方法在多个标准基准上取得当前最优(SOTA)性能,展现出优异泛化能力与精度。代码已开源:https://github.com/linliboabc-maker/stereo-matching-digital。
原文摘要 · Abstract (English)
Generally, monocular methods capture rich contextual priors but lack geometric precision, whereas stereo methods are geometrically accurate yet struggle in textureless and occluded regions. Several approaches attempt to combine their strengths to enhance the generalization of stereo matching (SM) by aligning monocular depth with stereo information. However, establishing a stable and generalizable alignment is challenging, and unreliable monocular cues can substantially degrade performance. This paper rethinks monocular depth embedding. First, to prevent shortcut learning, we reduce branch coupling instead of expanding network width. Second, we construct soft constraints instead of hard ones from monocular depth to improve tolerance to monocular depth errors. Based on the principles, we integrate monocular information into both feature extraction and GRU iterations. Specifically, the monocular depth map is fused with the RGB image to sharpen depth boundary perception and suppress matching ambiguities. The fused image is then used for feature extraction, allowing the contextual features to encode global geometric information. Furthermore, the monocular depth gradient feature is employed to guide disparity updates, helping to escape local oscillations. Finally, to address the boundary blurring of supervised disparity caused by data augmentation, we propose an edge confidence estimation method and an edge-aware loss function. Our method achieves state-of-the-art (SOTA) performance on multiple standard benchmarks, demonstrating excellent generalization while improving accuracy. The code is available at https://github.com/linliboabc-maker/stereo-matching-digital.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。