arXiv:2508.08679cs.CVcs.AI2025-08被引 1

提出自适应损失的多尺度可逆网络,提升多模态医学影像融合效果

MMIF-AMIN: Adaptive Loss-Driven Multi-Scale Invertible Dense Network for Multimodal Medical Image Fusion

  • 用可逆密集网络无损提取单模态特征
  • 多尺度互补特征模块融合不同模态信息,性能超越9种先进方法
  • 自适应损失函数提升数据挖掘深度,适合医学影像分析研究者

多模态医学图像融合(MMIF)旨在整合不同成像模态的图像,生成全面图像以增强医疗诊断,准确呈现器官结构、组织纹理和代谢信息。同时捕捉各模态的独特性与互补性是核心挑战。本文提出新型融合方法MMIF-AMIN,采用可逆密集网络(IDN)实现单模态特征的无损提取;设计多尺度互补特征提取模块(MCFEM),结合混合注意力机制、多尺寸卷积层与Transformer,有效挖掘模态间互补信息;引入自适应损失函数,克服传统人工设计损失的局限,深化数据挖掘。大量实验表明,MMIF-AMIN在定量与定性评估中均优于9种先进方法。消融实验证明各模块有效性。此外,该方法扩展至其他图像融合任务也取得良好表现。

原文摘要 · Abstract (English)

Multimodal medical image fusion (MMIF) aims to integrate images from different modalities to produce a comprehensive image that enhances medical diagnosis by accurately depicting organ structures, tissue textures, and metabolic information. Capturing both the unique and complementary information across multiple modalities simultaneously is a key research challenge in MMIF. To address this challenge, this paper proposes a novel image fusion method, MMIF-AMIN, which features a new architecture that can effectively extract these unique and complementary features. Specifically, an Invertible Dense Network (IDN) is employed for lossless feature extraction from individual modalities. To extract complementary information between modalities, a Multi-scale Complementary Feature Extraction Module (MCFEM) is designed, which incorporates a hybrid attention mechanism, convolutional layers of varying sizes, and Transformers. An adaptive loss function is introduced to guide model learning, addressing the limitations of traditional manually-designed loss functions and enhancing the depth of data mining. Extensive experiments demonstrate that MMIF-AMIN outperforms nine state-of-the-art MMIF methods, delivering superior results in both quantitative and qualitative analyses. Ablation experiments confirm the effectiveness of each component of the proposed method. Additionally, extending MMIF-AMIN to other image fusion tasks also achieves promising performance.

医学影像图像融合可逆网络多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。