arXiv:2605.20495cs.CV2026-05中稿 · CVPR

用主动学习优化显微图像提示选择,大幅减少专家标注量。

A Human-in-the-Loop Framework for Efficient Prompt Selection in Microscopy Vision-Language Models

  • 将提示集构建转为目标驱动的主动学习,优先选最有价值图像验证。
  • 仅需平均20个标注样本即可达到100%测试准确率。
  • 适合生物医学图像分析中希望降低标注成本的研究者。

显微图像分类的深度学习流程通常需要大量昂贵、耗时的专家标注来生成高质量训练标签。近期研究显示,通过视觉语言模型(VLM)的提示调优,可利用少量经专家验证的图像-标题样例作为少样本上下文,在推理时分类所有剩余图像,从而减少人工标注。为进一步降低成本,VLM可先生成候选样例的标题,由专家验证并轻量编辑,而非从零撰写。然而仍有两个关键问题未解决:(1) 应优先验证哪些未标注图像?(2) 需要多少已验证样例才能达到性能目标?本文通过将提示集构建建模为目标驱动的主动学习问题,回答上述问题。在小规模未标注数据池的严格低资源条件下,评估了三种互补的选择标准。实验表明,相较随机选择,本方法以更少的专家验证图像即达目标性能,平均仅需20个标注样本即可实现100%测试准确率。整体而言,该人机协同框架展示了生成式AI在生物医学图像分析中的以人为本应用,专家持续参与模型输出的验证与优化,同时显著降低标注成本。代码与数据将公开。

原文摘要 · Abstract (English)

Deep-learning pipelines for microscopy image classification often require expensive, labor- and time-intensive expert annotation to produce high-quality ground truth for training. Recent work has shown that prompt tuning of vision-language models (VLMs) can reduce manual annotation by constructing a small prompt set of expert-verified image-caption exemplars that is reused as few-shot context to classify all remaining images at inference time. To further reduce effort, the VLM can draft captions for candidate exemplars, which experts then verify and lightly edit instead of writing text de novo. However, two practical questions remain unaddressed: (1) which unlabeled images should be prioritized for verification, and (2) how many verified exemplars are needed to reach a performance target. In this work, we address these questions by formulating prompt-set construction as a target-driven active learning problem that prioritizes which images to annotate. We study three complementary selection criteria under strict low-resource constraints with small unlabeled pools. Experiments show that our methods reach the target performance with substantially fewer expert-verified images than random selection, achieving 100% test accuracy with as few as 20 annotated images on average. More broadly, our human-in-the-loop framework demonstrates a human-centered use of generative AI in biomedical image analysis, where experts remain actively involved in verifying and refining model output while significantly reducing annotation cost. Code and data will be publicly available.

主动学习显微图像提示调优人机协同

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。