通过深度图建模空间关系，提升交通场景语义分割精度
Depth Matters: Exploring Deep Interactions of RGB-D for Semantic Segmentation in Traffic Scenes
- 设计深度感知注意力机制，显式建模真实空间位置
- 在KITTI和Cityscapes上分别提升7.5%和4.9%的分割准确率
- 适合自动驾驶中需高精度深度融合的视觉任务
RGB-D数据在辅助驾驶复杂场景理解中日益重要，但现有研究忽视了深度图的内在空间特性，导致注意力表征偏差，引发预测错误。为此，我们提出可学习的深度交互金字塔变压器（DiPFormer），以挖掘深度信息的有效性。首先，引入深度空间感知优化（Depth SAO）作为偏移量，表征真实世界的空间关系；其次，通过深度线性交叉注意力（Depth LCA）学习RGB-D特征空间中的相似性，明确像素级空间差异；最后，采用MLP解码器有效融合多尺度特征，满足实时需求。大量实验表明，所提方法显著缓解了道路检测（+7.5%）与语义分割（+4.9%/+1.5%）中的注意力错位问题。DiPFormer在KITTI（道路F-score 97.57%，KITTI-360 mIoU 68.74%）和Cityscapes（mIoU 83.4%）数据集上均达到领先水平。
原文摘要 · Abstract (English)
RGB-D has gradually become a crucial data source for understanding complex scenes in assisted driving. However, existing studies have paid insufficient attention to the intrinsic spatial properties of depth maps. This oversight significantly impacts the attention representation, leading to prediction errors caused by attention shift issues. To this end, we propose a novel learnable Depth interaction Pyramid Transformer (DiPFormer) to explore the effectiveness of depth. Firstly, we introduce Depth Spatial-Aware Optimization (Depth SAO) as offset to represent real-world spatial relationships. Secondly, the similarity in the feature space of RGB-D is learned by Depth Linear Cross-Attention (Depth LCA) to clarify spatial differences at the pixel level. Finally, an MLP Decoder is utilized to effectively fuse multi-scale features for meeting real-time requirements. Comprehensive experiments demonstrate that the proposed DiPFormer significantly addresses the issue of attention misalignment in both road detection (+7.5%) and semantic segmentation (+4.9% / +1.5%) tasks. DiPFormer achieves state-of-the-art performance on the KITTI (97.57% F-score on KITTI road and 68.74% mIoU on KITTI-360) and Cityscapes (83.4% mIoU) datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。