提出多值分类的最优组合界,突破了传统方法的指数瓶颈。
An Optimal Sauer Lemma Over $k$-ary Alphabets
- 用DS维和列表DS维构建紧致的Sauer不等式
- 将列表大小ℓ的依赖从指数降为多项式,提升精度
- 适用于多值学习与列表预测,对理论研究者有价值
Sauer-Shelah-Perles引理是组合数学与学习理论的基石,用于界定二元假设类的大小与其Vapnik-Chervonenkis(VC)维度的关系。对于取值于k元字母表的函数类(即多类情形),长期以来以Natarajan维作为VC维的类比,但该类界限在k>2时并不紧致。本文建立了多类与列表预测的最优Sauer不等式,其形式基于Daniely-Shalev-Shwartz(DS)维及其扩展——列表DS维,这些组合参数刻画了多类与列表PAC可学习性。我们的界对任意字母表大小k、列表大小ℓ及维度值均紧致,将原有基于Natarajan维的指数级ℓ依赖替换为最优多项式依赖,并优化了对k的依赖关系。证明采用多项式方法。与经典VC情形中存在多个直接组合证明不同,我们尚不知晓任何纯组合性的DS情形证明,这为未来研究指明方向。作为应用,我们获得了列表PAC学习与列表预测一致收敛性的改进样本复杂度上界,优于Charikar等人(STOC 2023)、Hanneke等人(COLT 2024)及Brukhim等人(NeurIPS 2024)的最新结果。
原文摘要 · Abstract (English)
The Sauer-Shelah-Perles Lemma is a cornerstone of combinatorics and learning theory, bounding the size of a binary hypothesis class in terms of its Vapnik-Chervonenkis (VC) dimension. For classes of functions over a $k$-ary alphabet, namely the multiclass setting, the Natarajan dimension has long served as an analogue of VC dimension, yet the corresponding Sauer-type bounds are suboptimal for alphabet sizes $k>2$. In this work, we establish a sharp Sauer inequality for multiclass and list prediction. Our bound is expressed in terms of the Daniely--Shalev-Shwartz (DS) dimension, and more generally with its extension, the list-DS dimension -- the combinatorial parameters that characterize multiclass and list PAC learnability. Our bound is tight for every alphabet size $k$, list size $\ell$, and dimension value, replacing the exponential dependence on $\ell$ in the Natarajan-based bound by the optimal polynomial dependence, and improving the dependence on $k$ as well. Our proof uses the polynomial method. In contrast to the classical VC case, where several direct combinatorial proofs are known, we are not aware of any purely combinatorial proof in the DS setting. This motivates several directions for future research, which are discussed in the paper. As consequences, we obtain improved sample complexity upper bounds for list PAC learning and for uniform convergence of list predictors, sharpening the recent results of Charikar et al.~(STOC~2023), Hanneke et al.~(COLT~2024), and Brukhim et al.~(NeurIPS~2024).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。