arXiv:2604.00070eess.IVcs.AI2026-04

用单个T2w图像生成缺失的脑部MRI多模态影像,保留肿瘤特征。

Brain MR Image Synthesis with 3D Multi-Contrast Self-Attention GAN

论文配图:Brain MR Image Synthesis with 3D Multi-Contrast Self-Attention GAN
图 1 · 摘自论文原文
  • 基于3D自注意力机制的生成网络,统一合成多种MRI模态。
  • 生成图像在结构和肿瘤分布上与真实数据高度一致,保真度领先。
  • 适合需要减少扫描时间但又需完整病理信息的临床研究场景。

完整的高质多模态磁共振成像对神经肿瘤评估至关重要,每种对比度提供互补的解剖与病理信息。然而,因扫描时间长、成本高及患者不适,获取所有模态(如T1c、T1n、T2w、T2f)常不现实,可能影响全面肿瘤评估。本文提出3D-MC-SAGAN,一种统一的3D多模态合成框架,仅需一个T2w输入即可生成高质量的缺失模态,同时显式保留肿瘤特征。模型采用多尺度3D编码器-解码器生成器与残差连接,引入新型记忆受限混合注意力(MBHA)块以高效捕捉长程依赖;通过WGAN-GP判别器与辅助域分类头联合训练,统一生成T2f、T1n、T1c三维图像。为确保解剖与病理保真度,引入冻结的3D U-Net分割网络,在训练中施加肿瘤一致性约束。复合目标函数融合对抗损失、重建损失、感知损失、结构相似性、对比度分类与分割引导损失,促进全局真实感与肿瘤结构保持。在多个3D脑部MRI数据集上的实验表明,3D-MC-SAGAN达到当前最优定量性能,生成图像视觉连贯、解剖合理,且分布更接近真实数据。重要的是,该方法生成的肿瘤分割准确率与全模态输入相当,证明其可在降低采集负担的同时保留临床关键信息。

原文摘要 · Abstract (English)

Complete and high-quality multi-modal Magnetic Resonance Imaging (MRI) is essential for accurate neuro-oncological assessment, as each contrast provides complementary anatomical and pathological information. However, acquiring all modalities (e.g., T1c, T1n, T2w, T2f) for every patient is often impractical due to prolonged scan times, cost, and patient discomfort, potentially limiting comprehensive tumour evaluation. We propose 3D-MC-SAGAN (3D Multi-Contrast Self-Attention Generative Adversarial Network), a unified 3D multi-contrast synthesis framework that generates high-fidelity missing modalities from a single T2w input while explicitly preserving tumour characteristics. The model employs a multi-scale 3D encoder--decoder generator with residual connections and a novel Memory-Bounded Hybrid Attention (MBHA) block to capture long-range dependencies efficiently, and is trained with a WGAN-GP critic and an auxiliary domain classification head to produce T2f, T1n, and T1c volumes within a unified network. To ensure anatomical and pathological fidelity, we incorporate a frozen 3D U-Net-based segmentation network that enforces a tumour-consistency constraint during training. A composite objective combining adversarial, reconstruction, perceptual, structural similarity, contrast-classification, and segmentation-guided losses further promotes both global realism and tumour-preserving structure. Extensive experiments on 3D brain MRI datasets demonstrate that 3D-MC-SAGAN achieves state-of-the-art quantitative performance and produces visually coherent, anatomically plausible contrasts with improved distributional realism. Importantly, the proposed method maintains tumour segmentation accuracy comparable to that achieved using fully acquired multi-modal inputs, highlighting its potential to reduce acquisition burden while preserving clinically meaningful information.

医学影像图像合成生成模型脑肿瘤

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。