用自适应阈值提升伪标签质量,仅用1%标签实现顶尖图像分类效果。
SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning
- 根据模型学习进度动态调整每类阈值,避免固定阈值误差
- 在ImageNet-1K上仅用1%标签达80.7%准确率,10%标签超全监督模型
- 框架高效可扩展,适合资源受限下的高精度半监督学习场景
神经网络在有监督学习中表现优异,得益于大量高质量标注数据。然而,真实场景中获取此类数据成本高昂。半监督学习(SSL)为此提供解决方案。近期如Semi-ViT和Noisy Student等方法通过一致性正则化或伪标签法取得显著进展,但仍面临固定阈值导致高质量伪标签选择困难的问题。尽管FlexMatch和FreeMatch引入灵活或自适应阈值机制,其每轮迭代更新阈值过程耗时且计算开销大。为此,本文提出自训练自适应阈值(SST)框架,创新性地设计自适应阈值(SAT)机制,依据模型学习进度动态调整类别专属阈值,确保高质伪标签选取,缓解错误伪标签与确认偏差风险。大量实验表明,SST在多种架构与数据集上均达到最先进性能,兼具卓越效率、泛化性和可扩展性。在ImageNet-1K SSL基准测试中,仅使用1%/10%标签的Semi-SST-ViT-Huge分别取得80.7%/84.9%的Top-1准确率。相比之下,全监督DeiT-III-ViT-Huge使用100%标签仅达84.8%准确率,本方法以更少标签实现超越。
原文摘要 · Abstract (English)
Neural networks have demonstrated exceptional performance in supervised learning, benefiting from abundant high-quality annotated data. However, obtaining such data in real-world scenarios is costly and labor-intensive. Semi-supervised learning (SSL) offers a solution to this problem. Recent studies, such as Semi-ViT and Noisy Student, which employ consistency regularization or pseudo-labeling, have demonstrated significant achievements. However, they still face challenges, particularly in accurately selecting sufficient high-quality pseudo-labels due to their reliance on fixed thresholds. Recent methods such as FlexMatch and FreeMatch have introduced flexible or self-adaptive thresholding techniques, greatly advancing SSL research. Nonetheless, their process of updating thresholds at each iteration is deemed time-consuming, computationally intensive, and potentially unnecessary. To address these issues, we propose Self-training with Self-adaptive Thresholding (SST), a novel, effective, and efficient SSL framework. SST introduces an innovative Self-Adaptive Thresholding (SAT) mechanism that adaptively adjusts class-specific thresholds based on the model's learning progress. SAT ensures the selection of high-quality pseudo-labeled data, mitigating the risks of inaccurate pseudo-labels and confirmation bias. Extensive experiments demonstrate that SST achieves state-of-the-art performance with remarkable efficiency, generalization, and scalability across various architectures and datasets. Semi-SST-ViT-Huge achieves the best results on competitive ImageNet-1K SSL benchmarks, with 80.7% / 84.9% Top-1 accuracy using only 1% / 10% labeled data. Compared to the fully-supervised DeiT-III-ViT-Huge, which achieves 84.8% Top-1 accuracy using 100% labeled data, our method demonstrates superior performance using only 10% labeled data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。