无需提示词的3D神经显微图像分割新方法，提升复杂结构识别精度。
SAM4EM: Efficient memory-based two stage prompt-free segment anything model adapter for complex 3D neuroscience electron microscopy stacks
- 用两阶段解码自动生成提示嵌入，实现无提示词3D分割
- 低秩微调+3D记忆注意力，在少标注数据下显著提升精度
- 专为星形胶质突触等复杂结构设计，适合神经科学图像分析
我们提出SAM4EM，一种基于分割一切模型（SAM）并结合先进微调策略的新型3D神经结构分割方法。通过两阶段掩码解码实现无提示词的提示嵌入自动生成，采用基于低秩适应（LoRA）的双阶段微调以在有限标注数据下增强分割性能，并引入3D记忆注意力机制确保3D堆栈中分割的一致性。我们还发布了首个针对星形胶质突触与突触的专用基准数据集。在复杂的神经科学分割任务中，包括线粒体、胶质细胞和突触的分割，本方法在多个基准上显著优于现有最优（SOTA）方案，尤其在胶质细胞及突触后致密区等复杂结构上表现突出。代码与模型已公开于https://github.com/Uzshah/SAM4EM。
原文摘要 · Abstract (English)
We present SAM4EM, a novel approach for 3D segmentation of complex neural structures in electron microscopy (EM) data by leveraging the Segment Anything Model (SAM) alongside advanced fine-tuning strategies. Our contributions include the development of a prompt-free adapter for SAM using two stage mask decoding to automatically generate prompt embeddings, a dual-stage fine-tuning method based on Low-Rank Adaptation (LoRA) for enhancing segmentation with limited annotated data, and a 3D memory attention mechanism to ensure segmentation consistency across 3D stacks. We further release a unique benchmark dataset for the segmentation of astrocytic processes and synapses. We evaluated our method on challenging neuroscience segmentation benchmarks, specifically targeting mitochondria, glia, and synapses, with significant accuracy improvements over state-of-the-art (SOTA) methods, including recent SAM-based adapters developed for the medical domain and other vision transformer-based approaches. Experimental results indicate that our approach outperforms existing solutions in the segmentation of complex processes like glia and post-synaptic densities. Our code and models are available at https://github.com/Uzshah/SAM4EM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。