让医学图像分割模型学会判断问题真假，不瞎画
MedVeriSeg: Teaching LISA-Like Medical Segmentation Models to Verify Query Validity Without Extra Training
- 用相似性评分模块量化查询与图像的匹配度
- 通过多智能体融合判断，误报率降低47%以上
- 无需额外训练，适合临床部署的医疗视觉模型
尽管基于文本提示的医学图像分割取得进展，现有类似LISA的多模态大模型方法通常在目标不存在时仍生成掩码，导致幻觉分割。本文提出MedVeriSeg，一种无需训练的查询验证框架，使LISA类医学分割模型能够拒绝无效查询。该框架首先通过相似性响应质量评分模块，量化[SEG]标记与图像特征间的响应质量；为进一步提升鲁棒性，采用轻量级路由多智能体验证模块，融合定量评分与定性智能体证据，全面验证查询有效性。为支持系统评估，构建MedVeriSeg-Bench基准数据集。实验表明，MedVeriSeg能有效识别虚假查询，显著减少幻觉分割，同时保持对有效查询的高接受率，从而在不牺牲分割能力的前提下大幅增强模型可靠性。
原文摘要 · Abstract (English)
Despite recent progress in text-prompt-based medical image segmentation, existing LISA-like MLLM-based methods typically generate masks regardless of whether the target specified in the query is present, leading to hallucinated segmentation. In this work, we propose MedVeriSeg, a training-free query verification framework that enables LISA-like medical segmentation models to reject false segmentation queries. MedVeriSeg first quantifies the response quality between the [SEG] token and image features through a Similarity Response Quality Scoring Module. To further improve robustness, it employs a Lightweight Routed Multi-Agent Verification Module, which fuses quantitative score evidence with qualitative agent evidence to comprehensively verify the validity of the query. To support systematic evaluation, we construct MedVeriSeg-Bench, a benchmark designed for query verification in medical image segmentation. Experimental results demonstrate that MedVeriSeg effectively identifies false segmentation queries and reduces hallucinated segmentation, while maintaining a high acceptance rate for valid queries, thereby largely preserving the segmentation utility of LISA-like medical segmentation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。