对比学习提升多标签分类效果,尤其在标签多的数据上表现更优。
Multi-Label Contrastive Learning : A Comprehensive Study
- 采用监督对比学习构建标签相关性表征空间,增强模型对复杂标签关系的捕捉能力。
- 在标签数量多的数据集上,宏观F1得分显著优于传统方法。
- 适合处理标签复杂、数据量大的视觉与文本多标签任务。
多标签分类需为单个输入分配多个标签,在科研与工业中应用广泛。设计有效损失函数对优化深度神经网络至关重要,直接影响模型性能与效率。传统损失函数假设标签独立,难以捕捉复杂的标签关系。近期研究转向监督对比学习,通过拉近相似样本、推远不相似样本,构建结构化表征空间。尽管对比学习前景广阔,应用于多标签分类仍面临标签交互与数据结构管理难题。本文系统研究了多标签分类中对比学习损失在不同场景下的表现,涵盖标签数少与多、训练数据量大与小的多种数据集,以及计算机视觉与自然语言处理的应用。实验表明,对比学习的成功不仅源于对标签关系的建模,也得益于其稳健的优化机制。尤其在标签数量多的数据集上,该方法在宏平均F1指标上表现优异,而标签少时在排序类指标上仍存挑战。
原文摘要 · Abstract (English)
Multi-label classification, which involves assigning multiple labels to a single input, has emerged as a key area in both research and industry due to its wide-ranging applications. Designing effective loss functions is crucial for optimizing deep neural networks for this task, as they significantly influence model performance and efficiency. Traditional loss functions, which often maximize likelihood under the assumption of label independence, may struggle to capture complex label relationships. Recent research has turned to supervised contrastive learning, a method that aims to create a structured representation space by bringing similar instances closer together and pushing dissimilar ones apart. Although contrastive learning offers a promising approach, applying it to multi-label classification presents unique challenges, particularly in managing label interactions and data structure. In this paper, we conduct an in-depth study of contrastive learning loss for multi-label classification across diverse settings. These include datasets with both small and large numbers of labels, datasets with varying amounts of training data, and applications in both computer vision and natural language processing. Our empirical results indicate that the promising outcomes of contrastive learning are attributable not only to the consideration of label interactions but also to the robust optimization scheme of the contrastive loss. Furthermore, while the supervised contrastive loss function faces challenges with datasets containing a small number of labels and ranking-based metrics, it demonstrates excellent performance, particularly in terms of Macro-F1, on datasets with a large number of labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。