DB-SAM提升医学图像分割精度,解决自然图像模型在医疗领域表现差的问题。
DB-SAM: Delving into High Quality Universal Medical Image Segmentation
- 双分支结构:视觉变压器+卷积分支,分别提取深层与浅层特征。
- 3D医学分割任务上比现有方法提升8.8%平均性能。
- 适合需要高精度医学图像分割的研究者和临床应用开发者。
近期的通用图像分割模型SAM在多种下游任务中表现出色,但在通用医学图像分割中因自然图像与2D/3D医学数据间的领域差异,直接应用时性能显著下降。本文提出一种双分支自适应的SAM框架——DB-SAM,以有效弥合该领域差距。该框架包含并行的ViT分支与卷积分支:前者在每个冻结的注意力模块后加入可学习通道注意力块,捕捉特定领域局部特征;后者采用轻量卷积块从输入医学图像中提取浅层特征。为实现跨分支特征融合,设计了双边交叉注意力块与ViT-卷积融合块,动态整合双分支信息供掩码解码器使用。在大规模包含多种2D/3D医学分割任务的数据集上进行的大量实验表明,所提方法具有显著优势。在21个3D医学图像分割任务中,相比文献中最近的医学SAM适配器,本方法绝对提升8.8%。代码与模型已公开于https://github.com/AlfredQin/DB-SAM。
原文摘要 · Abstract (English)
Recently, the Segment Anything Model (SAM) has demonstrated promising segmentation capabilities in a variety of downstream segmentation tasks. However in the context of universal medical image segmentation there exists a notable performance discrepancy when directly applying SAM due to the domain gap between natural and 2D/3D medical data. In this work, we propose a dual-branch adapted SAM framework, named DB-SAM, that strives to effectively bridge this domain gap. Our dual-branch adapted SAM contains two branches in parallel: a ViT branch and a convolution branch. The ViT branch incorporates a learnable channel attention block after each frozen attention block, which captures domain-specific local features. On the other hand, the convolution branch employs a light-weight convolutional block to extract domain-specific shallow features from the input medical image. To perform cross-branch feature fusion, we design a bilateral cross-attention block and a ViT convolution fusion block, which dynamically combine diverse information of two branches for mask decoder. Extensive experiments on large-scale medical image dataset with various 3D and 2D medical segmentation tasks reveal the merits of our proposed contributions. On 21 3D medical image segmentation tasks, our proposed DB-SAM achieves an absolute gain of 8.8%, compared to a recent medical SAM adapter in the literature. The code and model are available at https://github.com/AlfredQin/DB-SAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。