通过平衡标签相关与区分关系,提升多标签图像分类效果。
Correlative and Discriminative Label Grouping for Multi-Label Visual Prompt Tuning
- 按共现和互斥关系分组标签,用提示词学习建模
- 在多个数据集上超越现有最优方法
- 适合需要精准多标签分类的研究者
多标签图像分类中建模标签相关性一直至关重要,但近期研究过度关注标签共现关系,易导致过拟合。为此,本文提出一种新型、参数高效的多标签视觉提示调优框架,通过标签共现与互斥关系将类别分组,并分别建模以平衡相关性与区分性。每个组内使用多个提示词令牌,捕获组内标签的相关或区分关系,有效学习子集级表示。同时,每组生成组感知视觉表征,结合专家混合(MoE)模型自适应地映射为标签感知表征,更利于分类。在多个基准数据集上的实验表明,该方法性能优异,优于多种预训练模型下的主流方法。
原文摘要 · Abstract (English)
Modeling label correlations has always played a pivotal role in multi-label image classification (MLC), attracting significant attention from researchers. However, recent studies have overemphasized co-occurrence relationships among labels, which can lead to overfitting risk on this overemphasis, resulting in suboptimal models. To tackle this problem, we advocate for balancing correlative and discriminative relationships among labels to mitigate the risk of overfitting and enhance model performance. To this end, we propose the Multi-Label Visual Prompt Tuning framework, a novel and parameter-efficient method that groups classes into multiple class subsets according to label co-occurrence and mutual exclusivity relationships, and then models them respectively to balance the two relationships. In this work, since each group contains multiple classes, multiple prompt tokens are adopted within Vision Transformer (ViT) to capture the correlation or discriminative label relationship within each group, and effectively learn correlation or discriminative representations for class subsets. On the other hand, each group contains multiple group-aware visual representations that may correspond to multiple classes, and the mixture of experts (MoE) model can cleverly assign them from the group-aware to the label-aware, adaptively obtaining label-aware representation, which is more conducive to classification. Experiments on multiple benchmark datasets show that our proposed approach achieves competitive results and outperforms SOTA methods on multiple pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。