不依赖标签也能训练出与监督学习相当的模型。
Is Supervised Learning Really That Different from Unsupervised?
- 先无标签训练模型参数，再加入标签输出，参数不变。
- 线性与核岭回归等模型性能接近传统监督方法。
- 适用于不想用标签或标签难获取的研究场景。
我们证明了监督学习可分解为两阶段过程：(1) 所有模型参数通过无监督方式选定；(2) 输出标签 y 被加入模型，但不改变已有参数。这一方法基于一种新模型选择准则，无需 y 即可使用。对线性岭回归，我们给出了该方法渐近泛化风险的上界，其与最优渐近风险相关。实验表明，线性与核岭回归、平滑样条、k-近邻、随机森林及神经网络在未使用标签的情况下训练，性能与标准监督方法相近。结果表明，监督与无监督学习的差异不如表面看来那般根本。
原文摘要 · Abstract (English)
We demonstrate how supervised learning can be decomposed into a two-stage procedure, where (1) all model parameters are selected in an unsupervised manner, and (2) the outputs y are added to the model, without changing the parameter values. This is achieved by a new model selection criterion that - in contrast to cross-validation - can be used also without access to y. For linear ridge regression, we bound the asymptotic out-of-sample risk of our method in terms of the optimal asymptotic risk. We also demonstrate that versions of linear and kernel ridge regression, smoothing splines, k-nearest neighbors, random forests, and neural networks, trained without access to y, perform similarly to their standard y-based counterparts. Hence, our results suggest that the difference between supervised and unsupervised learning is less fundamental than it may appear.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。