用基础模型提升肠镜中息肉检测,零样本也能超顶尖模型。
AI-Assisted Colonoscopy: Polyp Detection and Segmentation using Foundation Models
- 用五大基础模型对比检测与分割息肉,测试跨数据集泛化能力。
- 领域专精模型零样本表现优于微调后的传统模型,最高准确率94.3%。
- 适合医疗图像领域数据少场景,为无标注数据建模提供新路径。
在肠镜检查中,80%的遗漏息肉可通过深度学习模型发现。面对这一挑战,基础模型因其零样本或少样本学习能力,展现出在医学影像领域中的巨大潜力——该领域常缺乏大规模标注数据。本研究对五种基础模型(DINOv2、YOLO-World、GroundingDINO、SAM、MedSAM)在息肉检测与分割任务上的表现进行了全面评估,使用三个不同的肠镜数据集,对比其与两种基准模型(YOLOv8、Mask R-CNN)的性能。结果表明,基础模型在医学应用中的成功高度依赖于领域专精程度。仅通过领域特定微调,这些模型即能超越现有最先进模型;部分模型在未见数据上实现零样本评估时,甚至超过经过微调的基线模型,最高达到94.3%的分割精度。这证明领域适配是发挥基础模型优势的关键。
原文摘要 · Abstract (English)
In colonoscopy, 80% of the missed polyps could be detected with the help of Deep Learning models. In the search for algorithms capable of addressing this challenge, foundation models emerge as promising candidates. Their zero-shot or few-shot learning capabilities, facilitate generalization to new data or tasks without extensive fine-tuning. A concept that is particularly advantageous in the medical imaging domain, where large annotated datasets for traditional training are scarce. In this context, a comprehensive evaluation of foundation models for polyp segmentation was conducted, assessing both detection and delimitation. For the study, three different colonoscopy datasets have been employed to compare the performance of five different foundation models, DINOv2, YOLO-World, GroundingDINO, SAM and MedSAM, against two benchmark networks, YOLOv8 and Mask R-CNN. Results show that the success of foundation models in polyp characterization is highly dependent on domain specialization. For optimal performance in medical applications, domain-specific models are essential, and generic models require fine-tuning to achieve effective results. Through this specialization, foundation models demonstrated superior performance compared to state-of-the-art detection and segmentation models, with some models even excelling in zero-shot evaluation; outperforming fine-tuned models on unseen data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。