首次实现确定性苹果品鉴学习,解决长期未解难题。
Deterministic Apple Tasting
- 提出首个通用确定性算法,突破随机化限制。
- 可学习类的错误界为 $O( ext{L}( ext{H}) T /log T)$,且紧致。
- 区分易、难、不可学三类问题,适合理论学习者参考。
在二元在线分类的苹果品鉴反馈模型中,学习者仅在预测为1时获得反馈。此前所有已知算法均为随机化,因此确定性学习是否普遍可行仍未知。本文首次构建广泛适用的确定性苹果品鉴学习算法,并证明在可实现情形下,一个假设类可学习当且仅当其可确定性学习,验证了[Raman et al., 2024]的猜想。定量上,任意假设类$/mathcal{H}$的错误界为$Oig( ext{L}( ext{H}) T /log Tig)$(其中$ ext{L}( ext{H})$为Littlestone维数),且对某些类紧致。进一步研究非可实现情形,即最优假设最多犯$k$次错误,证明三分类定理:易类有随机与确定性错误界$Θ_{ ext{H}}(k)$;难类随机错误界为$ ildeΘ_{ ext{H}}(k + ext{T})$,确定性为$ ildeΘ_{ ext{H}}( ext{kT})$;不可学类两者错误界均为$Θ( ext{T})$。上界基于一个确定性专家建议学习算法,其最优确定性错误界为$Θig( ext{T}(k + ext{n})ig)$,适用于所有$ k, T ≤ n ≤ 2^T $,其中$ n $为专家数。
原文摘要 · Abstract (English)
In binary ($0/1$) online classification with apple tasting feedback, the learner receives feedback only when predicting $1$. Besides some degenerate learning tasks, all previously known learning algorithms for this model are randomized. Consequently, prior to this work it was unknown whether deterministic apple tasting is generally feasible. In this work, we provide the first widely-applicable deterministic apple tasting learner, and show that in the realizable case, a hypothesis class is learnable if and only if it is deterministically learnable, confirming a conjecture of [Raman, Subedi, Raman, Tewari-24]. Quantitatively, we show that every class $\mathcal{H}$ is learnable with mistake bound $O \left(\sqrt{\mathtt{L}(\mathcal{H}) T \log T} \right)$ (where $\mathtt{L}(\mathcal{H})$ is the Littlestone dimension of $\mathcal{H}$), and that this is tight for some classes. We further study the agnostic case, in which the best hypothesis makes at most $k$ many mistakes, and prove a trichotomy stating that every class $\mathcal{H}$ must be either easy, hard, or unlearnable. Easy classes have (both randomized and deterministic) mistake bound $Θ_{\mathcal{H}}(k)$. Hard classes have randomized mistake bound $\tildeΘ_{\mathcal{H}} \left(k + \sqrt{T} \right)$, and deterministic mistake bound $\tildeΘ_{\mathcal{H}} \left(\sqrt{k \cdot T} \right)$, where $T$ is the time horizon. Unlearnable classes have (both randomized and deterministic) mistake bound $Θ(T)$. Our upper bound is based on a deterministic algorithm for learning from expert advice with apple tasting feedback, a problem interesting in its own right. For this problem, we show that the optimal deterministic mistake bound is $Θ\left(\sqrt{T (k + \log n)} \right)$ for all $k$ and $T \leq n \leq 2^T$, where $n$ is the number of experts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。