解释了为何半监督学习比有监督学习泛化更好，从特征学习角度给出理论依据。
Towards Understanding Why FixMatch Generalizes Better Than Supervised Learning
- 对比分析发现：半监督学得更全的语义特征，有监督只随机学部分特征。
- 实验验证：新模型SA-FixMatch在多个数据集上提升泛化性能。
- 适合研究半监督学习机制、改进模型泛化能力的读者参考。
半监督学习（SSL）以FixMatch为代表，在深度神经网络（DNNs）上表现出显著优于有监督学习（SL）的泛化能力。然而，从理论上仍不清楚为何FixMatch类算法在DNN上能取得更好的测试准确率。本文首次针对卷积神经网络（CNN）在分类任务上的情况，提供了对FixMatch类方法泛化优势的理论解释。分析表明，FixMatch会学习每个语义类的所有判别性特征，而有监督学习由于彩票效应（lottery ticket hypothesis），仅随机捕获特征子集。此外，我们的分析框架可扩展至其他同类方法，如FlexMatch、FreeMatch、Dash和SoftMatch。基于此理论，我们提出了改进版FixMatch——语义感知修复匹配（SA-FixMatch）。实验结果证实了理论发现，并展示了SA-FixMatch更强的泛化能力。
原文摘要 · Abstract (English)
Semi-supervised learning (SSL), exemplified by FixMatch (Sohn et al., 2020), has shown significant generalization advantages over supervised learning (SL), particularly in the context of deep neural networks (DNNs). However, it is still unclear, from a theoretical standpoint, why FixMatch-like SSL algorithms generalize better than SL on DNNs. In this work, we present the first theoretical justification for the enhanced test accuracy observed in FixMatch-like SSL applied to DNNs by taking convolutional neural networks (CNNs) on classification tasks as an example. Our theoretical analysis reveals that the semantic feature learning processes in FixMatch and SL are rather different. In particular, FixMatch learns all the discriminative features of each semantic class, while SL only randomly captures a subset of features due to the well-known lottery ticket hypothesis. Furthermore, we show that our analysis framework can be applied to other FixMatch-like SSL methods, e.g., FlexMatch, FreeMatch, Dash, and SoftMatch. Inspired by our theoretical analysis, we develop an improved variant of FixMatch, termed Semantic-Aware FixMatch (SA-FixMatch). Experimental results corroborate our theoretical findings and the enhanced generalization capability of SA-FixMatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。