提出首个高效算法,可验证数据是否符合特定噪声假设。
Testing Noise Assumptions of Learning Algorithms
- 扩展测试学习框架,设计可验证噪声假设的检测机制。
- 在高斯分布上实现半空间带Massart噪声的多项式时间可测试学习。
- 揭示测试学习与经典学习的本质差异,复杂度显著更高。
我们提出了计算学习理论中的一个基本问题:能否高效检验训练集是否满足给定噪声模型的假设?尽管数十年来研究了噪声环境下的学习问题,该问题仍未得到解决。本文首次证明该任务是可 tractable 的,并提出首个针对多种噪声假设的高效检测算法。通过扩展 Rubinfeld 与 Vasilyan (2023) 提出的可测试学习框架,要求学习者运行一个满足两个条件的检测器:(1) 检测通过时,学习者输出分类器及最优性证书;(2) 对任意符合指定边缘分布与噪声模型的数据集,检测必须通过。我们研究在高斯边缘分布下,带有 Massart 噪声(每标签被翻转概率小于 1/2,且依赖于输入特征)的半空间学习问题,给出首个全多项式时间可测试学习算法。此外,我们揭示了经典结构化噪声学习与可测试学习之间的分离:对于简单的随机分类噪声(固定翻转概率 η=1/2),可测试学习需要超多项式时间,而经典学习则极为简单。
原文摘要 · Abstract (English)
We pose a fundamental question in computational learning theory: can we efficiently test whether a training set satisfies the assumptions of a given noise model? This question has remained unaddressed despite decades of research on learning in the presence of noise. In this work, we show that this task is tractable and present the first efficient algorithm to test various noise assumptions on the training data. To model this question, we extend the recently proposed testable learning framework of Rubinfeld and Vasilyan (2023) and require a learner to run an associated test that satisfies the following two conditions: (1) whenever the test accepts, the learner outputs a classifier along with a certificate of optimality, and (2) the test must pass for any dataset drawn according to a specified modeling assumption on both the marginal distribution and the noise model. We then consider the problem of learning halfspaces over Gaussian marginals with Massart noise (where each label can be flipped with probability less than $1/2$ depending on the input features), and give a fully-polynomial time testable learning algorithm. We also show a separation between the classical setting of learning in the presence of structured noise and testable learning. In fact, for the simple case of random classification noise (where each label is flipped with fixed probability $η= 1/2$), we show that testable learning requires super-polynomial time while classical learning is trivial.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。