用少量标注数据训练模型,自动生成医学图像分割提示,无需人工干预。
OFL-SAM2: Prompt SAM2 with Online Few-shot Learner for Efficient Medical Image Segmentation
- 用轻量映射网络从少量数据学医知识,自动生成目标特征
- 在线更新参数,测试时跨序列泛化能力强,准确率领先
- 适合标注稀缺的医疗场景,尤其适合资源有限的研究团队
Segment Anything Model 2 (SAM2) 在视频数据中展现出卓越的可提示分割能力,具备向医学图像分割(MIS)任务扩展的潜力,涵盖3D体数据和时间相关2D图像序列。然而,将SAM2应用于MIS面临挑战:需大量标注数据进行微调及高质量人工提示,均费时费力且依赖医学专家。为此,我们提出OFL-SAM2,一种标签高效的提示免扰医学图像分割框架。核心思路是利用有限标注样本训练一个轻量级映射网络,捕捉医学知识并将通用图像特征转化为目标特征,为每帧提供额外判别性表示,从而消除对人工提示的需求。关键在于,该映射网络支持推理时在线参数更新,提升模型在测试序列上的泛化能力。技术上引入两个关键组件:(1) 在线少样本学习器,使用有限数据训练映射网络生成目标特征;(2) 自适应融合模块,动态融合目标特征与冻结的SAM2生成的内存注意力特征,实现精准鲁棒的目标表征。在三个多样化的MIS数据集上的实验表明,OFL-SAM2以极少训练数据达到顶尖性能。
原文摘要 · Abstract (English)
The Segment Anything Model 2 (SAM2) has demonstrated remarkable promptable visual segmentation capabilities in video data, showing potential for extension to medical image segmentation (MIS) tasks involving 3D volumes and temporally correlated 2D image sequences. However, adapting SAM2 to MIS presents several challenges, including the need for extensive annotated medical data for fine-tuning and high-quality manual prompts, which are both labor-intensive and require intervention from medical experts. To address these challenges, we introduce OFL-SAM2, a prompt-free SAM2 framework for label-efficient MIS. Our core idea is to leverage limited annotated samples to train a lightweight mapping network that captures medical knowledge and transforms generic image features into target features, thereby providing additional discriminative target representations for each frame and eliminating the need for manual prompts. Crucially, the mapping network supports online parameter update during inference, enhancing the model's generalization across test sequences. Technically, we introduce two key components: (1) an online few-shot learner that trains the mapping network to generate target features using limited data, and (2) an adaptive fusion module that dynamically integrates the target features with the memory-attention features generated by frozen SAM2, leading to accurate and robust target representation. Extensive experiments on three diverse MIS datasets demonstrate that OFL-SAM2 achieves state-of-the-art performance with limited training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。