用高效微调技术让生物医学图像分割模型更省资源、更快适应新数据。
Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging
- 在SAM模型中插入轻量级微调层，仅调整少量参数提升性能。
- 实验表明微调层位置比类型对效率影响更大，可节省大量计算资源。
- 适合需要快速适配新生物图像数据的研究者，尤其关注算力受限场景。
生物医学图像分割是研究细胞器、细胞或器官的重要分析任务。深度学习虽显著提升了分割效果，但在新条件下泛化能力仍不足，需大量标注数据。视觉基础模型（如分割一切模型，SAM）通过更强的泛化能力缓解此问题。然而，为达到最优性能，仍需在标注数据上进行微调，尽管所需标注较少。其缺点是计算资源消耗大。因此，参数高效微调（PEFT）成为关键。本文首次系统研究了将PEFT应用于生物医学图像中的SAM模型。结果表明，对于视觉变换器而言，微调层的位置比类型对效率影响更大，并提出一套资源高效的微调方案。代码已公开于https://github.com/computational-cell-analytics/peft-sam。
原文摘要 · Abstract (English)
Segmentation is an important analysis task for biomedical images, enabling the study of individual organelles, cells or organs. Deep learning has massively improved segmentation methods, but challenges remain in generalization to new conditions, requiring costly data annotation. Vision foundation models, such as Segment Anything Model (SAM), address this issue through improved generalization. However, these models still require finetuning on annotated data, although with less annotations, to achieve optimal results for new conditions. As a downside, they require more computational resources. This makes parameter-efficient finetuning (PEFT) relevant. We contribute the first comprehensive study of PEFT for SAM applied to biomedical images. We find that the placement of PEFT layers is more important for efficiency than the type of layer for vision transformers and we provide a recipe for resource-efficient finetuning. Our code is publicly available at https://github.com/computational-cell-analytics/peft-sam.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。