揭示在线学习中未标记数据的真正价值,证明其能将错误率降低到√d量级。
Optimal Mistake Bounds for Transductive Online Learning
- 首次严格量化未标记数据在转导式在线学习中的增益效果
- 证明错误上界为O(√d),比旧下界提升指数级
- 适用于研究在线学习理论或算法设计的研究者
本文解决了在线学习领域一个持续30年的开放问题:未标记数据在转导式学习中的实际作用。标准在线学习的最优错误界限由概念类的Littlestone维数d决定(Littlestone, 1987)。我们证明,在转导式设置下,错误界限至少为Ω(√d)。这一结果相比此前Ω(log log d)、Ω(√log d)和Ω(log d)的下界实现了指数级提升,分别来自Ben-David、Kushilevitz、Mansour(1995, 1997)以及Hanneke、Moran、Shafer(2023)。同时我们证明该下界是紧的:对任意d,存在一个Littlestone维数为d的概念类,其转导式错误界为O(√d)。该上界也优于以往最佳上界(2/3)d(Ben-David et al., 1997)。这些结果确立了转导式与标准在线学习之间存在平方量级差距,凸显提前获知未标记样本序列的价值。这与PAC学习设定中两者样本复杂度相似形成对比。
原文摘要 · Abstract (English)
We resolve a 30-year-old open problem concerning the power of unlabeled data in online learning by tightly quantifying the gap between transductive and standard online learning. In the standard setting, the optimal mistake bound is characterized by the Littlestone dimension $d$ of the concept class $H$ (Littlestone 1987). We prove that in the transductive setting, the mistake bound is at least $Ω(\sqrt{d})$. This constitutes an exponential improvement over previous lower bounds of $Ω(\log\log d)$, $Ω(\sqrt{\log d})$, and $Ω(\log d)$, due respectively to Ben-David, Kushilevitz, and Mansour (1995, 1997) and Hanneke, Moran, and Shafer (2023). We also show that this lower bound is tight: for every $d$, there exists a class of Littlestone dimension $d$ with transductive mistake bound $O(\sqrt{d})$. Our upper bound also improves upon the best known upper bound of $(2/3)d$ from Ben-David, Kushilevitz, and Mansour (1997). These results establish a quadratic gap between transductive and standard online learning, thereby highlighting the benefit of advance access to the unlabeled instance sequence. This contrasts with the PAC setting, where transductive and standard learning exhibit similar sample complexities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。