让Mamba模型能处理任意形状缺失数据，提升视觉任务鲁棒性
Mask-aware inference with State-Space Models
- 引入局部掩码感知的重归一化机制，适配Mamba架构
- 在深度补全、图像修复等任务中显著优于传统方法
- 适用于含缺失数据的通用视觉任务，如遥感、医疗影像
许多真实场景的计算机视觉任务（如深度补全）需处理任意形状的缺失或无效数据区域。对于卷积神经网络（CNN），部分卷积通过仅基于有效像素进行掩码感知的重新归一化来解决该问题。近年来，状态空间模型（SSM）如Mamba因其线性复杂度和高性能而受到关注，但其在推理时缺乏对任意形状无效数据的原生处理能力。为此，我们提出部分视觉Mamba（PVM），将部分操作原理引入Mamba主干网络，并定义了使用PVM构建架构的一系列规则。实验表明，该方法在深度补全、图像修复以及含无效数据的分类任务中均表现出优异的性能与泛化能力。
原文摘要 · Abstract (English)
Many real-world computer vision tasks, such as depth completion, must handle inputs with arbitrarily shaped regions of missing or invalid data. For Convolutional Neural Networks (CNNs), Partial Convolutions solved this by a mask-aware re-normalization conditioned only on valid pixels. Recently, State Space Models (SSMs) like Mamba have emerged, offering high performance with linear complexity. However, these architectures lack an inherent mechanism for handling such arbitrarily shaped invalid data at inference time. To bridge this gap, we introduce Partial Vision Mamba (PVM), a novel architectural component that ports the principles of partial operations to the Mamba backbone. We also define a series of rules to design architectures using PVM. We show the efficacy and generalizability of our approach in the tasks of depth completion, image inpainting, and classification with invalid data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。