从可计算性角度解析PAC学习的构造难度,揭示不同信息下学习的复杂性本质。
Uniform Computability of PAC Learning
- 用Weihrauch复杂度分析不同信息表示下的PAC学习可计算性。
- 正向信息下,正确学习等价于巴尔空间极限操作;非正确学习则关联弱柯尼格引理。
- 结果揭示统计学习基本定理的构造程度,适合逻辑与计算复杂性研究者阅读。
本文利用Weihrauch复杂度研究PAC学习的统一可计算性,聚焦闭合概念类,其信息表示方式分别为正向、负向或完整信息。结果表明:基于正向信息的正确PAC学习等价于巴尔空间上的极限操作;非正确学习在有负向信息时等价于弱柯尼格引理;若允许任意假设,则仍处于有限阶非确定性不可计算(DNC)范围内,支持非确定算法但不支持概率算法。上述结论在提供VC维上界时均成立。进一步研究了当仅知VC维有限或采用负向/完整信息表示时的影响。还对VC维本身的计算复杂度进行了分类:正向或完整信息下等价于二元排序问题,负向信息下等价于排序的跳跃。该分类也揭示了PAC可学习性的Borel复杂度。
原文摘要 · Abstract (English)
We study uniform computability properties of PAC learning using Weihrauch complexity. We focus on closed concept classes, which are either represented by positive, by negative or by full information. Among other results, we prove that proper PAC learning from positive information is equivalent to the limit operation on Baire space, whereas improper PAC learning from positive information is closely related to Weak Kőnig's Lemma and even equivalent to it, when we have some negative information about the admissible hypotheses. If arbitrary hypotheses are allowed, then improper PAC learning from positive information is still in a finitary DNC range, which implies that it is non-deterministically computable, but does not allow for probabilistic algorithms. These results can also be seen as a classification of the degree of constructivity of the Fundamental Theorem of Statistical Learning. All the aforementioned results hold if an upper bound of the VC dimension is provided as an additional input information. We also study the question of how these results are affected if the VC dimension is not given, but only promised to be finite or if concept classes are represented by negative or full information. Finally, we also classify the complexity of the VC dimension operation itself, which is a problem that is of independent interest. For positive or full information it turns out to be equivalent to the binary sorting problem, for negative information it is equivalent to the jump of sorting. This classification allows also conclusions regarding the Borel complexity of PAC learnability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。