证明贪心教学集构造在小k时性能受限,暗示需研究高阶交互才能解决教学习题猜想。
Lower Bounds for Greedy Teaching Set Constructions
- 针对小k值分析贪心算法构建教学集的下界
- 当k=1时,性能不优于半分法的O(log|C|)上界
- 首次对k=2给出与已有上界匹配的下界,支持高阶交互必要性
学习理论中一个基础开放问题是如何刻画具有有限VC维d的概念类𝒞的最佳教学维度TS_min。解决此问题将直接验证Simon和Zilles(COLT 2015)提出的递归教学维度上界猜想。先前工作采用自然的贪心算法递归构造教学集,从而获得TS_min的上界,其中最优结果为O(d²)(Hu et al., COLT 2017)。该算法每轮选择使概念类缩小最多的k个标记样本加入教学集。本文证明该贪心方法在小k下的下界:当k=1时,其性能不优于基于半分法的O(log|𝒞|)上界;当k=2时,我们给出了与Moran等(FOCS 2015)O(log(log|𝒞|))上界相匹配的下界。最重要的是,我们的下界可扩展至k ≤ ⌈cd⌉(c>0为小常数),表明为解决TS_min = O(d)的猜想,研究更高阶交互是必要的。
原文摘要 · Abstract (English)
A fundamental open problem in learning theory is to characterize the best-case teaching dimension $\operatorname{TS}_{\min}$ of a concept class $\mathcal{C}$ with finite VC dimension $d$. Resolving this problem will, in particular, settle the conjectured upper bound on Recursive Teaching Dimension posed by [Simon and Zilles; COLT 2015]. Prior work used a natural greedy algorithm to construct teaching sets recursively, thereby proving upper bounds on $\operatorname{TS}_{\min}$, with the best known bound being $O(d^2)$ [Hu, Wu, Li, and Wang; COLT 2017]. In each iteration, this greedy algorithm chooses to add to the teaching set the $k$ labeled points that restrict the concept class the most. In this work, we prove lower bounds on the performance of this greedy approach for small $k$. Specifically, we show that for $k = 1$, the algorithm does not improve upon the halving-based bound of $O(\log(|\mathcal{C}|))$. Furthermore, for $k = 2$, we complement the upper bound of $O\left(\log(\log(|\mathcal{C}|))\right)$ from [Moran, Shpilka, Wigderson, and Yuhudayoff; FOCS 2015] with a matching lower bound. Most consequentially, our lower bound extends up to $k \le \lceil c d \rceil$ for small constant $c>0$: suggesting that studying higher-order interactions may be necessary to resolve the conjecture that $\operatorname{TS}_{\min} = O(d)$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。