结合几何模型与深度学习，提升立体视觉的精度与真实感。
Back to the Future Cyclopean Stereo: a human perception approach combining deep and geometric constraints
- 用双目视角模拟人类眼的3D表面建模，融合几何约束。
- 在遮挡和无纹理区域表现优于纯数据驱动方法。
- 适合虚拟现实与机器人感知，提升视觉真实性和安全性。
我们通过显式构建由人眼模型观测的3D表面解析模型，引入深度不连续性和遮挡信息，为立体视觉提供几何基础。该几何框架与学习到的立体特征相结合，充分发挥两种方法的优势。同时，利用先验单目表面模型填补遮挡或纹理缺失区域，解决匹配不足问题。实验结果已达到当前纯数据驱动方法的顶尖水平，且视觉质量显著更优，凸显3D几何模型对捕捉关键视觉信息的重要性。这种定性改进在虚拟现实领域可提升用户体验，在机器人领域有助于减少关键错误。本研究旨在证明理解并建模3D表面几何特性对计算机视觉研究具有重要价值。
原文摘要 · Abstract (English)
We innovate in stereo vision by explicitly providing analytical 3D surface models as viewed by a cyclopean eye model that incorporate depth discontinuities and occlusions. This geometrical foundation combined with learned stereo features allows our system to benefit from the strengths of both approaches. We also invoke a prior monocular model of surfaces to fill in occlusion regions or texture-less regions where data matching is not sufficient. Our results already are on par with the state-of-the-art purely data-driven methods and are of much better visual quality, emphasizing the importance of the 3D geometrical model to capture critical visual information. Such qualitative improvements may find applicability in virtual reality, for a better human experience, as well as in robotics, for reducing critical errors. Our approach aims to demonstrate that understanding and modeling geometrical properties of 3D surfaces is beneficial to computer vision research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。