arXiv:2410.00986eess.IVcs.CV2024-10被引 3

融合Transformer与CNN优势,提升高分辨率医学图像分割精度

TransResNet: Integrating the Strengths of ViTs and CNNs for High Resolution Medical Image Segmentation via Feature Grafting

  • 并行设计Transformer与CNN分支,分别提取多尺度特征
  • 通过交叉嫁接模块融合全局语义与局部细节,显著提升分割效果
  • 适用于皮肤病变、视网膜血管等高分辨率医学图像分割任务

高分辨率医学图像能显著提升诊断能力,尤其在自动图像分割中表现更优。然而,现有深度学习方法大多针对小尺寸输入优化,在高分辨率图像上表现不佳。为此,本文提出并行分支架构TransResNet,将Transformer与CNN并行结合,独立提取多分辨率特征。引入交叉嫁接模块(CGM),通过融合与自注意力机制,生成同时包含全局语义与低层空间细节的嫁接特征。这些特征用于解码过程,增强信息流动,提升分割掩码预测精度。在10个数据集上的大量实验表明,TransResNet在皮肤病变、视网膜血管和息肉分割等任务中均达到或接近当前最优性能。源代码与预训练模型已开源。

原文摘要 · Abstract (English)

High-resolution images are preferable in medical imaging domain as they significantly improve the diagnostic capability of the underlying method. In particular, high resolution helps substantially in improving automatic image segmentation. However, most of the existing deep learning-based techniques for medical image segmentation are optimized for input images having small spatial dimensions and perform poorly on high-resolution images. To address this shortcoming, we propose a parallel-in-branch architecture called TransResNet, which incorporates Transformer and CNN in a parallel manner to extract features from multi-resolution images independently. In TransResNet, we introduce Cross Grafting Module (CGM), which generates the grafted features, enriched in both global semantic and low-level spatial details, by combining the feature maps from Transformer and CNN branches through fusion and self-attention mechanism. Moreover, we use these grafted features in the decoding process, increasing the information flow for better prediction of the segmentation mask. Extensive experiments on ten datasets demonstrate that TransResNet achieves either state-of-the-art or competitive results on several segmentation tasks, including skin lesion, retinal vessel, and polyp segmentation. The source code and pre-trained models are available at https://github.com/Sharifmhamza/TransResNet.

医学图像分割TransformerCNN高分辨率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。