通过分组自适应交互提升多模态MRI图像合成效果
Learning Modality-Aware Representations: Adaptive Group-wise Interaction Network for Multimodal MRI Synthesis
- 将特征通道分组,用自适应卷积捕捉跨模态对应关系
- 在IXI和BraTS2023上实现当前最佳合成性能
- 适合医学影像生成、多模态融合研究者参考
多模态磁共振成像合成旨在通过融合部分可用模态,生成缺失模态的图像。现有方法多采用图像到图像翻译范式,将多模态作为输入通道处理,但因难以实现模态间精确的特征或语义对齐,常导致性能不佳。为此,本文提出自适应分组交互网络(AGI-Net),显式建模模态间与模态内关系。首先将特征通道划分为预定义组,再引入自适应滚动机制改进传统卷积核,以更好捕获不同模态间的特征与语义对应。同时,设计跨组注意力模块,促进组间特征融合,增强表示能力。在公开数据集IXI和BraTS2023上的实验表明,AGI-Net在多模态MRI合成任务中达到领先性能,验证了其模态感知交互设计的有效性。代码已开源。
原文摘要 · Abstract (English)
Multimodal MR image synthesis aims to generate missing modality images by effectively fusing and mapping from a subset of available MRI modalities. Most existing methods adopt an image-to-image translation paradigm, treating multiple modalities as input channels. However, these approaches often yield sub-optimal results due to the inherent difficulty in achieving precise feature- or semantic-level alignment across modalities. To address these challenges, we propose an Adaptive Group-wise Interaction Network (AGI-Net) that explicitly models both inter-modality and intra-modality relationships for multimodal MR image synthesis. Specifically, feature channels are first partitioned into predefined groups, after which an adaptive rolling mechanism is applied to conventional convolutional kernels to better capture feature and semantic correspondences between different modalities. In parallel, a cross-group attention module is introduced to enable effective feature fusion across groups, thereby enhancing the network's representational capacity. We validate the proposed AGI-Net on the publicly available IXI and BraTS2023 datasets. Experimental results demonstrate that AGI-Net achieves state-of-the-art performance in multimodal MR image synthesis tasks, confirming the effectiveness of its modality-aware interaction design. We release the relevant code at: https://github.com/zunzhumu/Adaptive-Group-wise-Interaction-Network-for-Multimodal-MRI-Synthesis.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。