arXiv:2409.08159cs.CV2024-09被引 5

用分块注意力机制的Transformer,高效完成深度补全任务。

SDformer: Efficient End-to-End Transformer for Depth Completion

  • 采用分块自注意力结构,避免全局计算开销
  • 在NYU Depth V2和KITTI DC上超越现有CNN模型
  • 参数与计算量更低,适合实时应用

深度补全旨在通过深度传感器的稀疏测量值预测稠密深度图。当前主流方法为基于卷积神经网络(CNN)的模型,虽性能优异,但特征表达范围受限。为克服CNN缺陷,本文提出一种新型窗口式Transformer架构——稀疏到稠密Transformer(SDformer),用于深度补全任务。网络包含输入模块(融合深度图与RGB图像特征)、基于U形结构的编码器-解码器Transformer模块以及精修模块。首先通过输入模块将深度特征与RGB特征拼接;随后,不直接对全图特征计算自注意力,而是采用不同窗口大小提取长距离深度依赖关系;最后,融合输入模块与主干网络的特征,并通过卷积层生成稠密深度图。实验表明,该方法在NYU Depth V2和KITTI DC数据集上优于现有CNN基线模型,且计算量与参数更少。

原文摘要 · Abstract (English)

Depth completion aims to predict dense depth maps with sparse depth measurements from a depth sensor. Currently, Convolutional Neural Network (CNN) based models are the most popular methods applied to depth completion tasks. However, despite the excellent high-end performance, they suffer from a limited representation area. To overcome the drawbacks of CNNs, a more effective and powerful method has been presented: the Transformer, which is an adaptive self-attention setting sequence-to-sequence model. While the standard Transformer quadratically increases the computational cost from the key-query dot-product of input resolution which improperly employs depth completion tasks. In this work, we propose a different window-based Transformer architecture for depth completion tasks named Sparse-to-Dense Transformer (SDformer). The network consists of an input module for the depth map and RGB image features extraction and concatenation, a U-shaped encoder-decoder Transformer for extracting deep features, and a refinement module. Specifically, we first concatenate the depth map features with the RGB image features through the input model. Then, instead of calculating self-attention with the whole feature maps, we apply different window sizes to extract the long-range depth dependencies. Finally, we refine the predicted features from the input module and the U-shaped encoder-decoder Transformer module to get the enriching depth features and employ a convolution layer to obtain the dense depth map. In practice, the SDformer obtains state-of-the-art results against the CNN-based depth completion models with lower computing loads and parameters on the NYU Depth V2 and KITTI DC datasets.

深度补全Transformer高效模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。