统一单目与立体深度估计，提升复杂表面的3D感知精度
BridgeDepth: Bridging Monocular and Stereo Reasoning with Latent Alignment
- 通过双向潜在空间对齐，让单目与立体模型互相补充信息
- 在Middlebury和ETH3D上零样本泛化误差降低40%以上
- 特别解决透明、反光等传统方法失效的难题，适合3D重建场景
单目深度估计擅长利用上下文先验，但几何精度不足；立体方法依赖视差几何，却难以处理反光或无纹理表面等模糊情况。尽管两者存在后融合协同，实际应用中仍相互割裂。本文提出统一框架，通过迭代双向潜空间对齐，实现单目与立体推理的深度融合。核心是新型交叉注意力对齐机制，在立体推理过程中动态同步单目上下文线索与立体假设表示，以单目结构先验缓解立体模糊（如镜面反射），同时用立体几何优化单目深度。大量实验表明，该方法在Middlebury和ETH3D数据集上实现超过40%的零样本泛化误差降低，显著改善对透明与反射表面的处理能力。通过融合多视角几何与单目上下文，本方法突破了模态固有局限，实现更鲁棒的3D感知。代码已开源：https://github.com/aeolusguan/BridgeDepth。
原文摘要 · Abstract (English)
Monocular and stereo depth estimation offer complementary strengths: monocular methods capture rich contextual priors but lack geometric precision, while stereo approaches leverage epipolar geometry yet struggle with ambiguities such as reflective or textureless surfaces. Despite post-hoc synergies, these paradigms remain largely disjoint in practice. We introduce a unified framework that bridges both through iterative bidirectional alignment of their latent representations. At its core, a novel cross-attentive alignment mechanism dynamically synchronizes monocular contextual cues with stereo hypothesis representations during stereo reasoning. This mutual alignment resolves stereo ambiguities (e.g., specular surfaces) by injecting monocular structure priors while refining monocular depth with stereo geometry within a single network. Extensive experiments demonstrate state-of-the-art results: \textbf{it reduces zero-shot generalization error by $\!>\!40\%$ on Middlebury and ETH3D}, while addressing longstanding failures on transparent and reflective surfaces. By harmonizing multi-view geometry with monocular context, our approach enables robust 3D perception that transcends modality-specific limitations. Codes available at https://github.com/aeolusguan/BridgeDepth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。