arXiv:2606.11626cs.CV2026-06IJCV

让视觉语言模型从只认主角转向识别多个对象,无需标签也能多标签分类。

Adapting Vision-Language Models from Iconic to Inclusive for Multi-Label Recognition Without Labels

论文配图:Adapting Vision-Language Models from Iconic to Inclusive for Multi-Label Recognition Without Labels
图 1 · 摘自论文原文
  • 通过分切-缝合两阶段无监督改造,避免模型只关注单一主体。
  • 在四个数据集上超越现有无监督方法,甚至接近弱监督基线性能。
  • 适合需要零样本多标签识别的场景,如开放世界图像理解。

多标签图像理解仍是计算机视觉中的挑战任务。随着视觉语言多模态学习的快速发展,视觉语言模型(VLMs)实现了无需标注数据的零样本识别。然而,由于其内在设计,这些模型往往优先关注最显著的物体,忽略其他上下文正例,这种固有偏见与多标签学习的本质相冲突,限制了其应用。本文提出一种无监督框架,将预训练的VLMs从“标志性识别”转向“包容性理解”,实现无标签的多标签图像识别。方法包含两个关键阶段:第一阶段“分切”引入多采样响应估计器,防止模型仅聚焦于单一对象;第二阶段“缝合”采用多对象融合适应策略,在仅一个训练周期内调整输出标签以符合多标签分布,同时保留原模型特性。大量实验表明,该框架在四个公开数据集上显著优于现有无监督方法,甚至超越若干代表性弱监督基线。结果证明了无需人工标注即可提升预训练VLMs的全面视觉理解能力。代码已开源:https://github.com/iCVTEAM/TailorCLIP。

原文摘要 · Abstract (English)

Understanding multi-label images remains a challenging task in computer vision. With the rapid progress of vision-language multimodal learning, vision-language models (VLMs) enable zero-shot recognition without labeled data. However, due to their intrinsic design, these models often prioritize the most iconic object and omit other contextual positives. This intrinsic bias conflicts with the nature of multi-label learning, thereby limiting their applicability. In this work, we propose an unsupervised framework that adapts VLMs from iconic recognition toward inclusive understanding, enabling label-free multi-label image recognition. Our approach consists of two key stages, ``cutting'' and ``sewing'': In the cutting stage, we present the multi-sampling response estimator to prevent the model from concentrating only on one single object. In the second sewing stage, the multi-object blend adaptation is introduced to adjust the labels to better conform to the multi-label distribution while preserving the intrinsic characteristics of the original model within only one epoch. Extensive experiments show that our framework significantly outperforms existing unsupervised approaches on four public datasets, even surpassing several representative weakly supervised baselines. These results demonstrate the potential of adapting pre-trained VLMs for more comprehensive visual understanding without manual annotations. Our code is publicly available at https://github.com/iCVTEAM/TailorCLIP.

多标签识别视觉语言模型无监督学习零样本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。