用多尺度融合Mamba网络,让低分辨率深度图更清晰。
Guided Depth Map Super-Resolution via Multi-Scale Fusion U-shaped Mamba Network
- 结合颜色图引导的多尺度融合结构,提升细节重建能力。
- 参数量更少,大尺寸深度图超分效果优于主流方法。
- 适合需要高效高精度深度图重建的视觉任务使用。
深度图超分辨率旨在提升低分辨率深度图的空间分辨率并有效恢复高频细节信息。传统卷积神经网络在处理长程依赖关系时存在局限,难以充分建模深度图中的全局上下文信息;尽管变压器能建模全局依赖,但其计算复杂度和内存消耗呈二次增长,严重限制了对高分辨率深度图的处理能力。本文提出一种多尺度融合U型Mamba(MSF-UM)模型,将Mamba的高效状态空间建模能力融入由彩色图像引导的多尺度U型融合结构中。设计了残差密集通道注意力块与Mamba状态空间模块相结合的结构,融合卷积层的局部特征提取能力与状态空间模型对远距离依赖的建模优势。同时,采用多尺度跨模态融合策略,充分利用彩色图像中的高频纹理信息来指导深度图的超分辨率过程。相比现有主流方法,所提模型显著减少参数量的同时实现了更高重建精度。在多个公开数据集上的大量实验验证了模型有效性,尤其在大规模深度图超分辨率任务中表现出优异的泛化能力。
原文摘要 · Abstract (English)
Depth map super-resolution technology aims to improve the spatial resolution of low-resolution depth maps and effectively restore high-frequency detail information. Traditional convolutional neural network has limitations in dealing with long-range dependencies and are unable to fully model the global contextual information in depth maps. Although transformer can model global dependencies, its computational complexity and memory consumption are quadratic, which significantly limits its ability to process high-resolution depth maps. In this paper, we propose a multi-scale fusion U-shaped Mamba (MSF-UM) model, a novel guided depth map super-resolution framework. The core innovation of this model is to integrate Mamba's efficient state-space modeling capabilities into a multi-scale U-shaped fusion structure guided by a color image. The structure combining the residual dense channel attention block and the Mamba state space module is designed, which combines the local feature extraction capability of the convolutional layer with the modeling advantage of the state space model for long-distance dependencies. At the same time, the model adopts a multi-scale cross-modal fusion strategy to make full use of the high-frequency texture information from the color image to guide the super-resolution process of the depth map. Compared with existing mainstream methods, the proposed MSF-UM significantly reduces the number of model parameters while achieving better reconstruction accuracy. Extensive experiments on multiple publicly available datasets validate the effectiveness of the model, especially showing excellent generalization ability in the task of large-scale depth map super-resolution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。