用自然语言精准分割3D医学影像,效率更高且细节更清晰。
ESICA: A Scalable Framework for Text-Guided 3D Medical Image Segmentation

- 基于相似矩阵的掩码预测,提升文本与图像语义对齐
- 轻量级解码器与两阶段优化,实现高精度体素分割
- 适合临床部署,小模型版本参数少但性能接近顶尖
文本引导的3D医学图像分割允许用户以自然语言直接指定感兴趣区域,摆脱预定义标签依赖,减少输出歧义,更贴合临床流程。然而现有方法普遍存在计算开销大、文本-图像特征对齐弱、难以捕捉细微解剖结构等问题。本文提出ESICA,通过三项创新解决:(1)基于相似矩阵的掩码预测机制,增强语义对齐;(2)高效的分解式解码器结合适配器模块,实现精确体积分割;(3)两阶段细化策略,锐化边界并处理不确定区域。为提升训练稳定性和泛化能力,采用仅正样本预训练+平衡微调的两阶段训练方案。在涵盖CT、MRI、PET、超声和显微镜五种成像模态的CVPR BiomedSegFM基准上,ESICA达到当前最优分割精度;其紧凑版ESICA4 Lite在参数显著减少的前提下仍保持相近性能,实现更优的效率-精度权衡。代码将公开于https://github.com/mirthAI/ESICA。
原文摘要 · Abstract (English)
Text guided 3D medical image segmentation offers a flexible alternative to class based and spatial prompt based models by allowing users to specify regions of interest directly in natural language. This paradigm avoids reliance on predefined label sets, reduces ambiguous outputs, and aligns more naturally with clinical workflows. However, existing text guided frameworks are often computationally expensive, exhibit weak text volume feature alignment, and fail to capture fine anatomical details. We propose ESICA, a lightweight and scalable framework that addresses these challenges through three innovations: (1) a similarity matrix based mask prediction formulation that enhances semantic alignment, (2) an efficient decomposed decoder with adapter modules for accurate volumetric decoding, and (3) a two pass refinement strategy that sharpens boundaries and resolves uncertain regions. To improve training stability and generalization, ESICA adopts a two stage scheme consisting of positive only pretraining followed by balanced fine tuning. On the CVPR BiomedSegFM benchmark spanning five imaging modalities (CT, MRI, PET, ultrasound, and microscopy), ESICA achieves state of the art segmentation accuracy, while the compact ESICA4 Lite variant attains similar segmentation performance with substantially fewer parameters, yielding a superior efficiency accuracy trade off. Our framework advances text guided segmentation toward efficient, scalable, and clinically deployable systems. Code will be made publicly available at https://github.com/mirthAI/ESICA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。