将弱文本提示转化为精准空间提示,提升医学图像分割精度。
PPBoost: Progressive Prompt Boosting for Text-Driven Medical Image Segmentation
- 用视觉语言模型生成初始伪框,再通过不确定性筛选优化
- 零样本下实现优于图文/视觉提示基线的分割效果
- 适合无标注数据场景,兼容多种主流分割模型
基于文本提示的医学图像分割模型虽操作直观,但空间精度不足且易受领域偏移影响。而依赖精确边界框的视觉提示模型性能强,但在临床中获取困难。本文提出PPBoost框架,在零样本条件下将弱文本信号转化为高精度空间提示:首先利用视觉语言模型生成初始伪框,并通过不确定性准则过滤不可靠预测;保留的图像-框对用于训练伪标签检测器,生成高质量边界框;推理时进一步适度扩展边界框以紧密覆盖目标结构。增强后的空间提示引导现有分割模型生成最终稠密掩码,显著放大弱文本信号。在三个涵盖不同模态和解剖结构的数据集上,PPBoost持续提升Dice和归一化表面距离指标,显著超越图文与视觉提示基线,且优于少样本分割模型,无需使用标注数据。该方法可适配多种主流分割模型主干网络。
原文摘要 · Abstract (English)
Text-prompted foundation models for medical image segmentation offer an intuitive way to delineate anatomical structures from natural language queries, but their predictions often lack spatial precision and degrade under domain shift. In contrast, visual-prompted models achieve strong segmentation performance across diverse modalities by leveraging spatial cues of precise bounding-box (bbox) prompts to guide the segmentation of target lesions. However, it is costly and challenging to obtain the precise visual prompts in clinical practice. We propose PPBoost (Progressive Prompt-Boosting), a framework that bridges these limitations by transforming weak text-derived signals into strong, spatially grounded visual prompts, operating under a strict zero-shot regime with no image- or pixel-level segmentation labels. PPBoost first uses a vision-language model to produce initial pseudo-bboxes conditioned on the textual object descriptions and applies an uncertainty-aware criterion to filter unreliable predictions. The retained image-bboxes pairs are then leveraged to train a pseudo-labeled detector, producing the high-quality bboxes for the query images. During inference, PPBoost further refines the generated bboxes by appropriately expanding them to tightly cover the target anatomical structures. The enhanced spatially-grounding bbox prompts guide existing segmentation models to generate final dense masks, effectively amplifying weak text cues into strong spatial guidance. Across three datasets spanning diverse modalities and anatomies, PPBoost consistently improves Dice and Normalized Surface Distance over text- and visual-prompted baselines and, notably, surpasses few-shot segmentation models without using labeled data. PPBoost can generalize to multiple typical visual segmentation model backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。