arXiv:2411.01399cs.CV2024-11被引 3

用Mamba模型解耦特征,实现无监督多模态图像精准配准

MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration

  • 基于Mamba架构解耦对齐与非对齐特征,提升可解释性
  • 在RGB-IR数据集上精度优于现有方法,变形场更平滑
  • 适合需要高精度且关注特征可解释性的医学图像配准场景

具有固有特征差异的多模态图像精确配准是形变图像配准中的关键挑战。传统学习方法常将注册网络视为黑箱,缺乏可解释性。核心洞察在于:解耦跨模态的对齐特征与非对齐特征具有优势。同时,主流图像配准方法如卷积神经网络受限于局部感受野,难以捕捉长程依赖,在大偏移情况下表现不佳。本文提出MambaReg,一种基于Mamba的新架构,利用其捕捉长序列的能力应对上述挑战。通过多个子模块设计,MambaReg能有效解耦负责配准的模态无关特征与模态相关、非对齐特征。通过选择性关注相关特征,网络能准确捕捉多模态图像间的关联,实现聚焦的形变场预测与精确图像对齐。该架构融合了卷积层的局部特征提取能力与Mamba的长程依赖建模能力。在公开的非刚性RGB-IR图像数据集上的实验表明,本方法在注册精度和形变场平滑性方面均优于现有方法。

原文摘要 · Abstract (English)

Precise alignment of multi-modal images with inherent feature discrepancies poses a pivotal challenge in deformable image registration. Traditional learning-based approaches often consider registration networks as black boxes without interpretability. One core insight is that disentangling alignment features and non-alignment features across modalities bring benefits. Meanwhile, it is challenging for the prominent methods for image registration tasks, such as convolutional neural networks, to capture long-range dependencies by their local receptive fields. The methods often fail when the given image pair has a large misalignment due to the lack of effectively learning long-range dependencies and correspondence. In this paper, we propose MambaReg, a novel Mamba-based architecture that integrates Mamba's strong capability in capturing long sequences to address these challenges. With our proposed several sub-modules, MambaReg can effectively disentangle modality-independent features responsible for registration from modality-dependent, non-aligning features. By selectively attending to the relevant features, our network adeptly captures the correlation between multi-modal images, enabling focused deformation field prediction and precise image alignment. The Mamba-based architecture seamlessly integrates the local feature extraction power of convolutional layers with the long-range dependency modeling capabilities of Mamba. Experiments on public non-rigid RGB-IR image datasets demonstrate the superiority of our method, outperforming existing approaches in terms of registration accuracy and deformation field smoothness.

图像配准Mamba多模态无监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。