arXiv:2602.04340cs.CVcs.AI2026-02

用双提示词显式建模不确定性,提升CLIP在少样本下的主动学习性能。

Explicit Uncertainty Modeling for Active CLIP Adaptation with Dual Prompt Tuning

  • 设计正负双提示词,分别增强分类判别力与建模预测置信度。
  • 在多个微调范式下,相同标注预算下准确率优于现有方法。
  • 适合资源受限场景中需要高效标注的视觉分类任务。

预训练视觉语言模型如CLIP具备强大迁移能力,但在标注预算有限的情况下将其适配到下游图像分类任务仍具挑战。在主动学习设置中,模型需从大量未标注数据中选出最具信息量的样本进行标注。现有方法通常通过熵或表示聚类估计不确定性,但未从模型角度显式建模。本文提出一种基于双提示词微调的稳健不确定性建模框架。在CLIP的文本分支引入两个可学习提示词:正提示词增强与轻量级微调视觉嵌入对应的任务特定文本嵌入的判别性,提升分类可靠性;负提示词以反向方式训练,显式建模预测标签正确的概率,为主动样本选择提供合理不确定性信号。在不同微调范式下的大量实验表明,该方法在相同标注预算下持续优于现有主动学习方法。

原文摘要 · Abstract (English)

Pre-trained vision-language models such as CLIP exhibit strong transferability, yet adapting them to downstream image classification tasks under limited annotation budgets remains challenging. In active learning settings, the model must select the most informative samples for annotation from a large pool of unlabeled data. Existing approaches typically estimate uncertainty via entropy-based criteria or representation clustering, without explicitly modeling uncertainty from the model perspective. In this work, we propose a robust uncertainty modeling framework for active CLIP adaptation based on dual-prompt tuning. We introduce two learnable prompts in the textual branch of CLIP. The positive prompt enhances the discriminability of task-specific textual embeddings corresponding to light-weight tuned visual embeddings, improving classification reliability. Meanwhile, the negative prompt is trained in an reversed manner to explicitly model the probability that the predicted label is correct, providing a principled uncertainty signal for guiding active sample selection. Extensive experiments across different fine-tuning paradigms demonstrate that our method consistently outperforms existing active learning methods under the same annotation budget.

主动学习CLIP提示词调优不确定性建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。