arXiv:2507.18082cs.CVcs.AI2025-07ICCV被引 10

用文本提示学习实现胰腺肿瘤自动分割,无需人工画框

TextSAM-EUS: Text Prompt Learning for SAM to Accurately Segment Pancreatic Tumor in Endoscopic Ultrasound

  • 基于BiomedCLIP和LoRA改造SAM,仅调参0.86%实现文本驱动分割
  • 自动提示下达82.69% Dice,手动提示下达83.10% Dice,超越现有模型
  • 适合医疗影像领域研究者,尤其关注少标注、自动化分割场景

胰腺癌预后差,依赖内镜超声(EUS)进行靶向活检和放疗。然而,EUS图像存在斑点噪声、对比度低、形态不直观等问题,导致全监督深度学习模型分割易出错,且依赖大量专家标注数据。为应对挑战,我们提出TextSAM-EUS,一种轻量级、文本驱动的Segment Anything Model(SAM)改进方法,推理时无需人工几何提示。该方法结合BiomedCLIP文本编码器与基于LoRA的SAM架构微调,仅调整总参数的0.86%,实现胰腺肿瘤在EUS中的自动分割。在公开的胰腺内镜超声数据库上,使用自动提示时取得82.69% Dice和85.28%标准化表面距离(NSD),使用手动几何提示时达83.10% Dice和85.70% NSD,优于现有最先进(SOTA)监督模型及基础模型(如SAM及其变体)。作为首个将提示学习引入SAM医疗图像分割的工作,TextSAM-EUS为高效、鲁棒的EUS分割提供了实用方案。代码已开源:https://github.com/HealthX-Lab/TextSAM-EUS。

原文摘要 · Abstract (English)

Pancreatic cancer carries a poor prognosis and relies on endoscopic ultrasound (EUS) for targeted biopsy and radiotherapy. However, the speckle noise, low contrast, and unintuitive appearance of EUS make segmentation of pancreatic tumors with fully supervised deep learning (DL) models both error-prone and dependent on large, expert-curated annotation datasets. To address these challenges, we present TextSAM-EUS, a novel, lightweight, text-driven adaptation of the Segment Anything Model (SAM) that requires no manual geometric prompts at inference. Our approach leverages text prompt learning (context optimization) through the BiomedCLIP text encoder in conjunction with a LoRA-based adaptation of SAM's architecture to enable automatic pancreatic tumor segmentation in EUS, tuning only 0.86% of the total parameters. On the public Endoscopic Ultrasound Database of the Pancreas, TextSAM-EUS with automatic prompts attains 82.69% Dice and 85.28% normalized surface distance (NSD), and with manual geometric prompts reaches 83.10% Dice and 85.70% NSD, outperforming both existing state-of-the-art (SOTA) supervised DL models and foundation models (e.g., SAM and its variants). As the first attempt to incorporate prompt learning in SAM-based medical image segmentation, TextSAM-EUS offers a practical option for efficient and robust automatic EUS segmentation. Code is available at https://github.com/HealthX-Lab/TextSAM-EUS .

医学图像文本提示胰腺肿瘤SAM改进

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。