提出高效多模态医学图像分析框架,提升精度与抗干扰能力。
Effective and Robust Multimodal Medical Image Analysis
- 设计双模块网络,分别捕捉模态特异性特征和跨模态互补信息。
- 在20个数据集上最高提升9.34%准确率,计算成本降低78.3%。
- 增强对抗鲁棒性,适合资源受限且需高可靠性的医疗场景。
多模态融合学习(MFL)利用多种影像模态(如MRI、CT、SPECT)的数据,在皮肤癌和脑肿瘤预测等医学问题中展现出巨大潜力。然而,现有方法存在三大局限:一、专精特定模态,忽视跨模态共享互补信息,限制多病种泛化能力;二、依赖高计算开销模型,难以在资源受限环境下应用;三、对对抗攻击缺乏鲁棒性,影响医疗AI可靠性。为此,我们提出新型多注意力集成学习(MAIL)网络,包含两个关键组件:其一为高效残差注意力块,用于捕捉精细化的模态特异性多尺度模式;其二为高效跨模态注意力模块,用于学习丰富互补的共享表征。为进一步保障对抗鲁棒性,我们扩展MAIL为鲁棒型MAIL(Robust-MAIL),引入随机投影滤波器与调制注意力噪声。在20个公开数据集上的广泛评估表明,MAIL与Robust-MAIL均优于现有方法,性能提升最高达9.34%,计算成本降低最高达78.3%。结果凸显了本方法的优势,确保预测更可靠。代码已开源:https://github.com/misti1203/MAIL-Robust-MAIL。
原文摘要 · Abstract (English)
Multimodal Fusion Learning (MFL), leveraging disparate data from various imaging modalities (e.g., MRI, CT, SPECT), has shown great potential for addressing medical problems such as skin cancer and brain tumor prediction. However, existing MFL methods face three key limitations: a) they often specialize in specific modalities, and overlook effective shared complementary information across diverse modalities, hence limiting their generalizability for multi-disease analysis; b) they rely on computationally expensive models, restricting their applicability in resource-limited settings; and c) they lack robustness against adversarial attacks, compromising reliability in medical AI applications. To address these limitations, we propose a novel Multi-Attention Integration Learning (MAIL) network, incorporating two key components: a) an efficient residual learning attention block for capturing refined modality-specific multi-scale patterns and b) an efficient multimodal cross-attention module for learning enriched complementary shared representations across diverse modalities. Furthermore, to ensure adversarial robustness, we extend MAIL network to design Robust-MAIL by incorporating random projection filters and modulated attention noise. Extensive evaluations on 20 public datasets show that both MAIL and Robust-MAIL outperform existing methods, achieving performance gains of up to 9.34% while reducing computational costs by up to 78.3%. These results highlight the superiority of our approaches, ensuring more reliable predictions than top competitors. Code: https://github.com/misti1203/MAIL-Robust-MAIL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。