用不确定性感知权重提升伪标签质量,解决阈值难题
Uncertainty-aware Long-tailed Weights Model the Utility of Pseudo-labels for Semi-supervised Learning
- 引入不确定性感知集成结构,动态评估伪标签可信度
- 采用长尾权重机制,使低置信度伪标签仍可贡献模型鲁棒性
- 轻量通用,适用于分类与回归任务,尤其在小样本下效果显著
当前半监督学习采用伪标签策略,并基于置信度阈值筛选。但该方法存在两大问题:1)合理阈值设定尚无定论,显著影响高质量伪标签选择;2)深度模型常出现过度自信现象,导致置信度不可靠,尤其在标注数据稀缺时。本文提出不确定性感知集成结构(UES),用于评估未标记样本伪标签的效用,并将伪标签效用建模为长尾权重,避免阈值设定难题。长尾权重机制确保即使不可靠的伪标签也能增强模型鲁棒性。此外,UES轻量且与架构无关,可轻松扩展至各类视觉任务,包括分类与回归。实验表明,结合DualPose,在Sniffing数据集上(100样本,30标注)PCK提升3.47%;在FLIC数据集上(100样本,50标注)提升7.29%;在LSP数据集上(200样本,100标注)提升3.91%。结合FixMatch,在CIFAR-10(40标注)上准确率提升0.2%,在CIFAR-100(400标注)上提升0.26%。
原文摘要 · Abstract (English)
Current Semi-supervised Learning (SSL) adopts the pseudo-labeling strategy and further filters pseudo-labels based on confidence thresholds. However, this mechanism has notable drawbacks: 1) setting the reasonable threshold is an open problem which significantly influences the selection of the high-quality pseudo-labels; and 2) deep models often exhibit the over-confidence phenomenon which makes the confidence value an unreliable indicator for assessing the quality of pseudo-labels due to the scarcity of labeled data. In this paper, we propose an Uncertainty-aware Ensemble Structure (UES) to assess the utility of pseudo-labels for unlabeled samples. We further model the utility of pseudo-labels as long-tailed weights to avoid the open problem of setting the threshold. Concretely, the advantage of the long-tailed weights ensures that even unreliable pseudo-labels still contribute to enhancing the model's robustness. Besides, UES is lightweight and architecture-agnostic, easily extending to various computer vision tasks, including classification and regression. Experimental results demonstrate that combining the proposed method with DualPose leads to a 3.47% improvement in Percentage of Correct Keypoints (PCK) on the Sniffing dataset with 100 data points (30 labeled), a 7.29\% improvement in PCK on the FLIC dataset with 100 data points (50 labeled), and a 3.91% improvement in PCK on the LSP dataset with 200 data points (100 labeled). Furthermore, when combined with FixMatch, the proposed method achieves a 0.2% accuracy improvement on the CIFAR-10 dataset with 40 labeled data points and a 0.26% accuracy improvement on the CIFAR-100 dataset with 400 labeled data points.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。