提出一种轻量级网络,高效融合多模态医学影像并增强边缘细节。
Edge-Enhanced Dilated Residual Attention Network for Multimodal Medical Image Fusion
- 用空洞残差注意力模块提取多尺度特征,结合梯度算子强化边缘
- 采用无需参数的加权核范数融合策略,推理速度极快且不增加计算量
- 在脑肿瘤分类任务中表现优于主流方法,适合临床实时应用
多模态医学图像融合能将不同成像模态的互补信息整合为统一表征,提升诊断准确性和治疗规划。尽管深度学习方法(如卷积神经网络和Transformer)显著提升了融合性能,但现有基于CNN的方法在捕捉细粒度多尺度与边缘特征方面仍有不足,导致特征融合效果不佳。而基于Transformer的模型在训练和融合阶段计算开销大,难以满足临床实时需求。此外,融合图像的临床应用尚未被充分探索。本文提出一种新型基于CNN的架构,通过引入空洞残差注意力模块实现有效多尺度特征提取,并结合梯度算子增强边缘细节学习。为保证快速高效融合,设计了一种无参的融合策略,基于softmax加权核范数,训练与推理均不引入额外计算开销。大量实验(包括下游脑肿瘤分类任务)表明,该方法在视觉质量、纹理保持和融合速度上均优于多种基线模型,具备实际临床应用潜力。代码将在https://github.com/simonZhou86/en_dran发布。
原文摘要 · Abstract (English)
Multimodal medical image fusion is a crucial task that combines complementary information from different imaging modalities into a unified representation, thereby enhancing diagnostic accuracy and treatment planning. While deep learning methods, particularly Convolutional Neural Networks (CNNs) and Transformers, have significantly advanced fusion performance, some of the existing CNN-based methods fall short in capturing fine-grained multiscale and edge features, leading to suboptimal feature integration. Transformer-based models, on the other hand, are computationally intensive in both the training and fusion stages, making them impractical for real-time clinical use. Moreover, the clinical application of fused images remains unexplored. In this paper, we propose a novel CNN-based architecture that addresses these limitations by introducing a Dilated Residual Attention Network Module for effective multiscale feature extraction, coupled with a gradient operator to enhance edge detail learning. To ensure fast and efficient fusion, we present a parameter-free fusion strategy based on the weighted nuclear norm of softmax, which requires no additional computations during training or inference. Extensive experiments, including a downstream brain tumor classification task, demonstrate that our approach outperforms various baseline methods in terms of visual quality, texture preservation, and fusion speed, making it a possible practical solution for real-world clinical applications. The code will be released at https://github.com/simonZhou86/en_dran.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。