Membership queries不能让可测试学习更快,最多只比纯样本学习快多项式倍。
Limitations of Membership Queries in Testable Learning
- 通过归约将拒绝学习转化为带查询的可测试学习,证明查询无本质加速作用。
- 在给定样本数下,带查询的可测试学习算法时间复杂度不会超多项式优于纯样本学习。
- 适用于关注学习效率与查询优势的理论学习研究者。
Membership queries (MQ) 常能加速学习任务,尤其在分布特定设置下。本文表明,在 Rubinfeld 与 Vasilyan [RV23] 提出的可测试学习模型中,成员查询无法使学习算法的时间复杂度低于仅使用样本的分布特定学习复杂度。在该模型中,当数据分布满足目标性质时,学习者必须输出假设;若输出,则假设必须近似最优。我们给出了从基于样本的布尔概念类拒绝学习(refutation)到带查询的可测试学习(TL-Q)的一般归约。由此,利用 [KL18] 中从学习到拒绝的归约,可得 TL-Q 的下界。结果表明:相对于某个概念类和分布族,任何 $m$-样本的 TL-Q 算法,其时间效率不会比最优 $m$-样本 PAC 学习器超多项式地更优。最后,我们定义了一类“统计性”成员查询算法,涵盖许多已知的分布特定学习器(如基于影响估计或子立方体条件统计查询的方法)。我们证明,此类中的 TL-Q 算法蕴含高效统计查询拒绝与学习算法。结合已知的统计查询维度下界,这表明这些高效的成员查询学习器无法被构造为可测试版本。
原文摘要 · Abstract (English)
Membership queries (MQ) often yield speedups for learning tasks, particularly in the distribution-specific setting. We show that in the \emph{testable learning} model of Rubinfeld and Vasilyan [RV23], membership queries cannot decrease the time complexity of testable learning algorithms beyond the complexity of sample-only distribution-specific learning. In the testable learning model, the learner must output a hypothesis whenever the data distribution satisfies a desired property, and if it outputs a hypothesis, the hypothesis must be near-optimal. We give a general reduction from sample-based \emph{refutation} of boolean concept classes, as presented in [Vadhan17, KL18], to testable learning with queries (TL-Q). This yields lower bounds for TL-Q via the reduction from learning to refutation given in [KL18]. The result is that, relative to a concept class and a distribution family, no $m$-sample TL-Q algorithm can be super-polynomially more time-efficient than the best $m$-sample PAC learner. Finally, we define a class of ``statistical'' MQ algorithms that encompasses many known distribution-specific MQ learners, such as those based on influence estimation or subcube-conditional statistical queries. We show that TL-Q algorithms in this class imply efficient statistical-query refutation and learning algorithms. Thus, combined with known SQ dimension lower bounds, our results imply that these efficient membership query learners cannot be made testable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。