让SAM模型精准分割癌细胞核,无需全量微调。
Adapting SAM to Nuclei Instance Segmentation and Classification via Cooperative Fine-Grained Refinement
- 用轻量级模块增强SAM对细胞核局部结构的感知能力。
- 三组件协同提升边界精度,实现亚像素级分割效果。
- 适合病理图像分析、医学图像分割研究者快速部署。
细胞核实例分割在计算病理学中对癌症诊断与预后至关重要。近期,基于自然图像大规模预训练的通用分割模型SAM展现出卓越性能,但直接应用于医学影像存在局限:对局部结构感知不足,且全量微调计算成本高昂。为此,我们提出一种参数高效微调框架——协同细粒度精炼SAM(Cooperative Fine-Grained Refinement of SAM),包含三个核心组件:1)多尺度自适应局部感知适配器,通过动态生成多尺度卷积核,在冻结SAM主干基础上以极少参数注入局部结构感知能力;2)分层调制融合模块,动态聚合多层级编码特征以保留精细空间细节;3)边界引导掩码精炼模块,显式融合多上下文边界线索与语义特征,生成边界聚焦信号,优化初始掩码预测以实现更清晰的轮廓分割。三者协同增强局部感知、保持空间细节并精炼边界,使SAM可直接实现高精度细胞核实例分割。
原文摘要 · Abstract (English)
Nuclei instance segmentation is critical in computational pathology for cancer diagnosis and prognosis. Recently, the Segment Anything Model has demonstrated exceptional performance in various segmentation tasks, leveraging its rich priors and powerful global context modeling capabilities derived from large-scale pre-training on natural images. However, directly applying SAM to the medical imaging domain faces significant limitations: it lacks sufficient perception of the local structural features that are crucial for nuclei segmentation, and full fine-tuning for downstream tasks requires substantial computational costs. To efficiently transfer SAM's robust prior knowledge to nuclei instance segmentation while supplementing its task-aware local perception, we propose a parameter-efficient fine-tuning framework, named Cooperative Fine-Grained Refinement of SAM, consisting of three core components: 1) a Multi-scale Adaptive Local-aware Adapter, which enables effective capability transfer by augmenting the frozen SAM backbone with minimal parameters and instilling a powerful perception of local structures through dynamically generated, multi-scale convolutional kernels; 2) a Hierarchical Modulated Fusion Module, which dynamically aggregates multi-level encoder features to preserve fine-grained spatial details; and 3) a Boundary-Guided Mask Refinement, which integrates multi-context boundary cues with semantic features through explicit supervision, producing a boundary-focused signal to refine initial mask predictions for sharper delineation. These three components work cooperatively to enhance local perception, preserve spatial details, and refine boundaries, enabling SAM to perform accurate nuclei instance segmentation directly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。