轻量级改进SAM模型，精准分割医学图像边界。
BALR-SAM: Boundary-Aware Low-Rank Adaptation of SAM for Resource-Efficient Medical Image Segmentation
- 引入边界感知的低秩适配框架，聚焦医学图像特征。
- 仅更新1.8%参数（1170万），分割精度超全微调模型。
- 内存降低75%，适合资源受限的临床部署场景。
视觉基础模型如基于大规模自然图像预训练的Segment Anything Model（SAM），在医学图像分割任务中因缺乏领域适应性而表现不佳。临床实践中，如何以极低资源开销高效微调此类模型，同时保持高性能，仍具挑战。为此，本文提出BALR-SAM：一种边界感知的低秩适配框架，用于优化SAM在医学影像中的表现。其包含三个定制模块：(1) 使用深度可分离卷积与多尺度融合的互补细节增强网络（CDEN），捕捉对精确分割至关重要的边界敏感特征；(2) 在SAM的视觉变换器块中集成低秩适配器，优化医学上下文下的特征表示与注意力机制，显著压缩参数量；(3) 在掩码解码器中引入低秩张量注意力机制，内存使用减少75%，推理速度提升。在标准医学分割数据集上的实验表明，BALR-SAM无需提示即可超越多个先进方法（包括全微调的MedSAM），仅更新1.8%（1170万）参数即达成优异性能。
原文摘要 · Abstract (English)
Vision foundation models like the Segment Anything Model (SAM), pretrained on large-scale natural image datasets, often struggle in medical image segmentation due to a lack of domain-specific adaptation. In clinical practice, fine-tuning such models efficiently for medical downstream tasks with minimal resource demands, while maintaining strong performance, is challenging. To address these issues, we propose BALR-SAM, a boundary-aware low-rank adaptation framework that enhances SAM for medical imaging. It combines three tailored components: (1) a Complementary Detail Enhancement Network (CDEN) using depthwise separable convolutions and multi-scale fusion to capture boundary-sensitive features essential for accurate segmentation; (2) low-rank adapters integrated into SAM's Vision Transformer blocks to optimize feature representation and attention for medical contexts, while simultaneously significantly reducing the parameter space; and (3) a low-rank tensor attention mechanism in the mask decoder, cutting memory usage by 75% and boosting inference speed. Experiments on standard medical segmentation datasets show that BALR-SAM, without requiring prompts, outperforms several state-of-the-art (SOTA) methods, including fully fine-tuned MedSAM, while updating just 1.8% (11.7M) of its parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。