arXiv:2502.03777cs.CV2025-02ICLR被引 14

针对多标签测试时自适应,提出绑定熵最小化方法提升多个标签预测置信度。

Multi-Label Test-Time Adaptation with Bound Entropy Minimization

  • 通过绑定前k个预测标签,联合优化图像与文本提示的置信度。
  • 在MSCOCO、VOC、NUSWIDE上优于当前最先进方法,跨模型架构稳定有效。
  • 适合处理标签数量不固定的多标签场景,尤其对弱标签数据敏感的任务。

主流测试时自适应(TTA)方法通过熵最小化缓解分布偏移,但仅提升最高置信度类别的概率,难以应对多标签任务中每张图像标签数不一的问题。为此,本文研究多标签测试时自适应(ML-TTA),提出边界熵最小化(BEM)目标,同时提升多个高置信度标签的置信度。具体地,对每个增强视图,检索其对应图文描述并提取文本标签,构建大小为k的弱标签集和强标签集。随后,BEM将图像与文本的前k个预测标签视为单一实体,联合学习图像与文本提示。通过绑定前k个标签,BEM克服了传统熵最小化只优化最可信类别的问题。在MSCOCO、VOC和NUSWIDE三个多标签数据集上,配备BEM的ML-TTA框架在多种模型架构、提示初始化方式及不同标签数量场景下均表现优于最新SOTA方法。

原文摘要 · Abstract (English)

Mainstream test-time adaptation (TTA) techniques endeavor to mitigate distribution shifts via entropy minimization for multi-class classification, inherently increasing the probability of the most confident class. However, when encountering multi-label instances, the primary challenge stems from the varying number of labels per image, and prioritizing only the highest probability class inevitably undermines the adaptation of other positive labels. To address this issue, we investigate TTA within multi-label scenario (ML--TTA), developing Bound Entropy Minimization (BEM) objective to simultaneously increase the confidence of multiple top predicted labels. Specifically, to determine the number of labels for each augmented view, we retrieve a paired caption with yielded textual labels for that view. These labels are allocated to both the view and caption, called weak label set and strong label set with the same size k. Following this, the proposed BEM considers the highest top-k predicted labels from view and caption as a single entity, respectively, learning both view and caption prompts concurrently. By binding top-k predicted labels, BEM overcomes the limitation of vanilla entropy minimization, which exclusively optimizes the most confident class. Across the MSCOCO, VOC, and NUSWIDE multi-label datasets, our ML--TTA framework equipped with BEM exhibits superior performance compared to the latest SOTA methods, across various model architectures, prompt initialization, and varying label scenarios. The code is available at https://github.com/Jinx630/ML-TTA.

多标签测试时自适应熵最小化视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。