用轻量方法让大模型高效分割细胞,不重训主干网络
Rethinking the Adaptation of Vision Foundation Models for Efficient Cell Segmentation

- 冻结大模型主干,用提示词编码器提取结构先验
- 仅500万参数,性能超越全量微调方法
- 适合标注少、算力有限的生物图像分析场景
细胞分割对计算病理学和生物发现至关重要。尽管近期视觉基础模型(VFMs)展现出强大的通用特征表示能力,但其在细胞图像中的潜力受限于资源密集型的适配方式。现有方法通常需微调庞大的视觉编码器,带来巨大计算开销并依赖大规模标注。为此,我们提出EffiCell-Seg框架,实现无需重训练视觉编码器的高效细胞分割。核心思想是:预训练的VFMs内在包含互补的结构先验——全局显著性用于定位潜在细胞,局部形态模式用于勾勒细胞边界。我们设计了细胞结构提示编码器(CSP-Encoder),将冻结的VFM表征中的语义显著性和主要形态特征合成显式的结构先验图。同时,提出协同掩码解码器(SM-Decoder),通过几何距离场与语义图的相互引导,强制上下文一致性。大量实验表明,EffiCell-Seg在多种细胞成像模态下均优于当前最优方法,且仅需约500万可训练参数,比全量微调的VFM方案减少超过130倍。代码已开源:https://github.com/xq141839/EffiCell-Seg。
原文摘要 · Abstract (English)
Cell segmentation is critical for computational pathology and biomedical discovery. While recent Vision Foundation Models (VFMs) have demonstrated remarkable universal feature representations, unlocking their full potential for cellular imaging is currently bottlenecked by resource-intensive adaptation paradigms. Existing methods typically rely on fine-tuning heavy visual encoders, leading to extensive computational overhead and a dependency on large-scale annotations. To address this, we propose the EffiCell-Seg framework for highly efficient cell segmentation without re-training the visual encoder. Our core insight is that pretrained VFMs intrinsically encode complementary structural priors: global saliency for localizing potential cells, and local morphological patterns for delineating cellular structures. To harness these priors, we devise a Cell Structure Prompt Encoder (CSP-Encoder) that synthesizes semantic-aware saliency and principal morphological features from frozen VFM representations into explicit structural prior maps. Moreover, we propose a Synergistic Mask Decoder (SM-Decoder) that enforces contextual consistency by jointly predicting geometric distance fields and semantic maps via mutual cross-guidance. Extensive experiments demonstrate that EffiCell-Seg outperforms state-of-the-art methods across diverse cell imaging modalities while requiring only ~5M trainable parameters, over 130x fewer than fully fine-tuned VFM counterparts. The code is available at https://github.com/xq141839/EffiCell-Seg.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。