通过结构评估提升无标签数据质量,让半监督学习更可靠
USE: Uncertainty Structure Estimation for Robust Semi-Supervised Learning
- 用代理模型计算无标签数据熵值,区分有用与无用样本
- 在CIFAR-100和Yelp数据上,提升准确率并增强对异常数据的鲁棒性
- 适合关注真实场景中半监督学习稳定性的研究者
本文提出一种轻量级、算法无关的不确定性结构估计(USE)方法,用于半监督学习(SSL)。现有SSL虽进展显著,但其部署可靠性受限于无标签数据质量。实际中,无标签数据常混入分布外(OOD)样本,近域与远域OOD均以不同方式损害性能。我们指出瓶颈不在于算法设计,而在于缺乏评估与筛选无标签数据质量的系统方法。USE利用标注集训练代理模型,计算无标签样本的熵值,并通过统计对比参考分布确定阈值,将样本划分为有结构(信息丰富)与无结构(无意义)两类。该方法可作为预处理步骤,在训练前剔除有害或无效数据。在图像(CIFAR-100)与自然语言(Yelp Review)数据上的大量实验表明,该方法在不同程度的OOD污染下均能持续提升准确率与鲁棒性。因此,该方法将无标签数据质量控制重新定义为结构性评估问题,认为其是真实混合分布环境中可靠高效半监督学习的必要组成部分。
原文摘要 · Abstract (English)
In this study, a novel idea, Uncertainty Structure Estimation (USE), a lightweight, algorithm-agnostic procedure that emphasizes the often-overlooked role of unlabeled data quality is introduced for Semi-supervised learning (SSL). SSL has achieved impressive progress, but its reliability in deployment is limited by the quality of the unlabeled pool. In practice, unlabeled data are almost always contaminated by out-of-distribution (OOD) samples, where both near-OOD and far-OOD can negatively affect performance in different ways. We argue that the bottleneck does not lie in algorithmic design, but rather in the absence of principled mechanisms to assess and curate the quality of unlabeled data. The proposed USE trains a proxy model on the labeled set to compute entropy scores for unlabeled samples, and then derives a threshold, via statistical comparison against a reference distribution, that separates informative (structured) from uninformative (structureless) samples. This enables assessment as a preprocessing step, removing uninformative or harmful unlabeled data before SSL training begins. Through extensive experiments on imaging (CIFAR-100) and NLP (Yelp Review) data, it is evident that USE consistently improves accuracy and robustness under varying levels of OOD contamination. Thus, it can be concluded that the proposed approach reframes unlabeled data quality control as a structural assessment problem, and considers it as a necessary component for reliable and efficient SSL in realistic mixed-distribution environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。