arXiv:2409.14113eess.IVcs.CV2024-09中稿 · as a poster by Med…被引 9

用频域与空间联合学习,加速多对比MRI重建

Accelerated Multi-Contrast MRI Reconstruction via Frequency and Spatial Mutual Learning

  • 通过频域-空间双分支提取特征,捕捉跨模态全局依赖
  • 在BraTS和fastMRI上实现最优重建性能,加速因子达8×
  • 适合医学影像加速重建、多模态融合研究者参考

为加速磁共振成像流程,多对比MR重建(MCMR)已成为主流趋势,利用易获取的模态作为辅助,以欠采样k空间数据高保真重建目标模态。探索不同模态间的全局依赖与互补信息对MCMR至关重要。然而现有方法或因感受野有限难以捕捉全局依赖,或面临二次计算复杂度问题。为此,本文提出新型频域与空间互学习网络(FSMNet),高效挖掘跨模态全局依赖。具体地,各模态特征由频域-空间特征提取(FSFE)模块提取,包含频域分支与空间分支:频域分支借助傅里叶变换的全局特性,实现图像级感受野的全局依赖捕捉;空间分支则提取局部特征。为利用辅助模态的互补信息,提出跨模态选择性融合(CMS-fusion)模块,有选择地融合辅助模态的频域与空间特征以增强目标模态对应分支。进一步,设计频域-空间融合(FS-fusion)模块,整合增强后的全局与局部特征,生成目标模态的完整表征。在BraTS与fastMRI数据集上的大量实验表明,所提FSMNet在不同加速因子下均达到当前最优的MCMR性能。代码已开源:https://github.com/qic999/FSMNet。

原文摘要 · Abstract (English)

To accelerate Magnetic Resonance (MR) imaging procedures, Multi-Contrast MR Reconstruction (MCMR) has become a prevalent trend that utilizes an easily obtainable modality as an auxiliary to support high-quality reconstruction of the target modality with under-sampled k-space measurements. The exploration of global dependency and complementary information across different modalities is essential for MCMR. However, existing methods either struggle to capture global dependency due to the limited receptive field or suffer from quadratic computational complexity. To tackle this dilemma, we propose a novel Frequency and Spatial Mutual Learning Network (FSMNet), which efficiently explores global dependencies across different modalities. Specifically, the features for each modality are extracted by the Frequency-Spatial Feature Extraction (FSFE) module, featuring a frequency branch and a spatial branch. Benefiting from the global property of the Fourier transform, the frequency branch can efficiently capture global dependency with an image-size receptive field, while the spatial branch can extract local features. To exploit complementary information from the auxiliary modality, we propose a Cross-Modal Selective fusion (CMS-fusion) module that selectively incorporate the frequency and spatial features from the auxiliary modality to enhance the corresponding branch of the target modality. To further integrate the enhanced global features from the frequency branch and the enhanced local features from the spatial branch, we develop a Frequency-Spatial fusion (FS-fusion) module, resulting in a comprehensive feature representation for the target modality. Extensive experiments on the BraTS and fastMRI datasets demonstrate that the proposed FSMNet achieves state-of-the-art performance for the MCMR task with different acceleration factors. The code is available at: https://github.com/qic999/FSMNet.

MRI重建多模态学习频域建模医学影像

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。