arXiv:2512.10244cs.CVcs.LG2025-12中稿 · ECCV被引 1

用温度调优让大模型更好利用无标签数据,提升少样本自标注性能。

Solving Semi-Supervised Few-Shot Learning from an Auto-Annotation Perspective

  • 通过调节softmax温度,增强伪标签置信度,提升无标签数据利用率。
  • 在五个基准上比最新少样本与自监督方法高约5个百分点。
  • 适合需要高效利用海量无标签数据的自动标注场景。

半监督少样本学习(SSFSL)模拟真实世界的自动标注场景,旨在仅用少量有标签和大量无标签的任务特定样本训练模型以标注未标记数据。尽管存在强大的开源视觉-语言模型(VLM)和开放世界数据,现有研究大多忽视这些资源。相比之下,少样本学习(FSL)已成功利用它们提升性能。为解决真实自动标注问题,我们探索使用标准自监督学习(SSL)方法微调VLM,但意外发现其性能显著低于不使用无标签数据的FSL基线。深入分析揭示根本原因:VLM生成的softmax概率分布平坦,导致无标签数据完全无法利用,且监督信号弱。为此,我们提出一种极简方案——使用温度参数锐化softmax输出,不仅提高伪标签置信度以增强无标签数据利用,还强化训练监督信号以实现有效微调。此外,我们引入任务相关的开放数据(如从VLM预训练集检索的数据),并通过分阶段训练策略缓解数据不平衡与领域差异。基于成功微调VLM和开放数据利用,我们提出简单高效的SSFSL方法:分阶段温度微调(SWIFT)。在五个基准上,SWIFT相比近期FSL与SSL方法提升约5个百分点,甚至媲美监督学习(假设无标签数据有真值标签)。

原文摘要 · Abstract (English)

Semi-supervised few-shot learning (SSFSL) resembles real-world applications such as auto-annotation, as it aims to learn a model from a few labeled and abundant unlabeled task-specific examples to annotate the unlabeled ones. Despite the availability of powerful open-source Vision-Language Models (VLMs) and open-world data, existing SSFSL literature largely neglects these resources. In contrast, the related area few-shot learning (FSL) has already exploited them to boost performance. Arguably, to solve real-world auto-annotation, SSFSL should leverage such open resources. To bridge this gap, we explore established SSL methods to finetune a VLM. Unexpectedly, they significantly underperform FSL baselines that do not use unlabeled data. Our in-depth analysis reveals the root cause of failure: VLMs produce flat distributions of softmax probabilities, resulting in zero utilization of unlabeled data and weak supervision signals. To address this challenge, we propose an embarrassingly simple solution that uses temperatures to sharpen the softmax output, which not only increases the confidence scores of pseudo-labels to improve the utilization of unlabeled data, but also strengthens training supervision for effective finetuning. Furthermore, we exploit task-relevant open data, e.g., those retrieved from VLMs' publicly available pretraining set. To mitigate the imbalance and domain gaps in retrieved data, we employ a stage-wise training strategy. Building on the successful finetuning of VLMs and the exploitation of open data, we present a simple yet effective SSFSL method, Stage-Wise Finetuning with Temperatures (SWIFT). Across five benchmarks, SWIFT outperforms recent FSL and SSL methods by $\sim$5 accuracy points. SWIFT even rivals supervised learning, which finetunes a VLM assuming unlabeled data having ground-truth labels!

少样本学习自监督视觉语言模型自动标注

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。