用分割特征提升立体深度估计,让模型更关注物体边界和纹理。
USAM-Net: A U-Net-based Network for Improved Stereo Correspondence and Scene Depth Estimation using Features from a Pre-trained Image Segmentation network
- 双路径结构融合预训练分割模型与立体图像,增强特征表达。
- 在DrivingStereo数据集上达到GD 3.61%、EPE 0.88的优异性能。
- 适合自动驾驶和增强现实等需要高精度深度的应用场景。
自动驾驶与增强现实对高精度深度估计的需求日益增长,亟需能有效利用多模态数据的神经网络架构。本文提出统一分割注意力机制网络(USAM-Net),一种基于U-Net的卷积神经网络,通过融合立体图像输入与语义分割图及注意力机制,提升深度估计性能。USAM-Net采用双路径架构,其中分割路径使用预训练模型(SAM)处理立体图像以生成语义掩码,并将其与原始图像拼接后输入深度估计路径。该设计使模型能够聚焦于物体边界和表面纹理等关键特征,显著改善深度感知能力。在DrivingStereo数据集上的实证评估表明,USAM-Net在全局差异(GD)达3.61%,端点误差(EPE)为0.88,优于传统模型如CFNet、SegStereo和iResNet。结果验证了将分割信息融入立体深度估计的有效性,凸显了USAM-Net在高精度深度应用中的潜力。
原文摘要 · Abstract (English)
The increasing demand for high-accuracy depth estimation in autonomous driving and augmented reality applications necessitates advanced neural architectures capable of effectively leveraging multiple data modalities. In this context, we introduce the Unified Segmentation Attention Mechanism Network (USAM-Net), a novel convolutional neural network that integrates stereo image inputs with semantic segmentation maps and attention to enhance depth estimation performance. USAM-Net employs a dual-pathway architecture, which combines a pre-trained segmentation model (SAM) and a depth estimation model. The segmentation pathway preprocesses the stereo images to generate semantic masks, which are then concatenated with the stereo images as inputs to the depth estimation pathway. This integration allows the model to focus on important features such as object boundaries and surface textures which are crucial for accurate depth perception. Empirical evaluation on the DrivingStereo dataset demonstrates that USAM-Net achieves superior performance metrics, including a Global Difference (GD) of 3.61\% and an End-Point Error (EPE) of 0.88, outperforming traditional models such as CFNet, SegStereo, and iResNet. These results underscore the effectiveness of integrating segmentation information into stereo depth estimation tasks, highlighting the potential of USAM-Net in applications demanding high-precision depth data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。