提出新框架,让监督学习者只需比可验证的半监督方法强,突破了以往理论瓶颈。
Relatively Smart: A New Approach for Instance-Optimal Learning
- 设计相对智能学习框架,只与可验证的半监督方法竞争
- 在无分布假设下,一包含图学习器样本复杂度仅需平方提升
- 适用于对理论性能有严格要求的研究者,尤其关注学习保证的可靠性
我们重新审视智能 PAC 学习框架,该框架旨在设计能与已知标签边缘分布的半监督学习者竞争的监督学习者。先前工作表明,对于“大多数”边缘分布(相对于某个固定已知测度而言),此类边际-边际保证是可能的,但无法普遍成立。我们发现失败源于一种“不可区分性”现象:某些边缘分布无法被统计区分,而它们需要不同的学习策略。在此类情形下,半监督学习无法从无标签数据中验证其保证,使其实际应用性存疑。为此,我们提出相对智能学习,要求监督学习者仅需超越最佳‘可验证’的半监督保证。我们证明,这种适度放宽足以绕过先前工作的不可能性结果。在无分布假设设定下,我们证明一包含图学习器在样本复杂度平方意义下是相对智能的,并且任何监督学习算法都无法更优。对于分布族情形,我们发现相对智能学习可能不可能,或需要特定学习方式,且其难度在分布族包含序上并非单调。
原文摘要 · Abstract (English)
We revisit the framework of Smart PAC learning, which seeks supervised learners which compete with semi-supervised learners that are provided full knowledge of the marginal distribution on unlabeled data. Prior work has shown that such marginal-by-marginal guarantees are possible for "most" marginals, with respect to an arbitrary fixed and known measure, but not more generally. We discover that this failure can be attributed to an "indistinguishability" phenomenon: There are marginals which cannot be statistically distinguished from other marginals that require different learning approaches. In such settings, semi-supervised learning cannot certify its guarantees from unlabeled data, rendering them arguably non-actionable. We propose relatively smart learning, a new framework which demands that a supervised learner compete only with the best "certifiable" semi-supervised guarantee. We show that such modest relaxation suffices to bypass the impossibility results from prior work. In the distribution-free setting, we show that the One-Inclusion Graph learner is relatively smart up to squaring the sample complexity, and show that no supervised learning algorithm can do better. For distribution-family settings, we show that relatively smart learning can be impossible or can require idiosyncratic learning approaches, and its difficulty can be non-monotone in the inclusion order on distribution families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。