重新定义学习稳定性,让更复杂的模型也能被有效学习
Stability and List-Replicability for Agnostic Learners
- 提出依赖误差差距的稳定性新标准,突破原有限制
- 证明无限模型类在该标准下仍可学习,关键指标是Littlestone维数
- 适用于研究在线学习与稳定学习关系的理论学者
两篇奠基性论文(Alon et al., STOC 2019;Bun et al., FOCS 2020)建立了二分类中在线可学习性与全局稳定PAC可学习性的等价性。然而,Chase et al.(STOC 2024)近期指出,在异议设置下该等价性不成立:只有有限假设类才是全局稳定可学习的。为此,他们提出了两种对异议全局稳定性的松弛。本文完全刻画了在这些松弛条件下可学习的假设类,解决了其工作中提出的两个开放问题。首先,当稳定性参数可依赖于超出误差(即学习器误差与假设类最优误差之差)时,异议稳定性由Littlestone维数完全刻画,因此该形式的可学习性再次等价于在线可学习性。作为证明的一部分,我们强化了Bun et al.的经典结果:即使允许稳定性参数依赖于超出误差,具有无限Littlestone维数的类仍不可稳定PAC学习。其次,对于Chase等人提出的第二种松弛,我们证明即便限定在总体损失较小的分布上,仅有有限假设类是全局稳定可学习的。
原文摘要 · Abstract (English)
Two seminal papers--Alon, Livni, Malliaris, Moran (STOC 2019) and Bun, Livni, and Moran (FOCS 2020)--established the equivalence between online learnability and globally stable PAC learnability in binary classification. However, Chase, Chornomaz, Moran, and Yehudayoff (STOC 2024) recently showed that this equivalence does not hold in the agnostic setting. Specifically, they proved that in the agnostic setting, only finite hypothesis classes are globally stable learnable. Therefore, agnostic global stability is too restrictive to capture interesting hypothesis classes. To address this limitation, Chase et al. introduced two relaxations of agnostic global stability. In this paper, we characterize the classes that are learnable under their proposed relaxed conditions, resolving the two open problems raised in their work. First, we prove that in the setting where the stability parameter can depend on the excess error (the gap between the learner's error and the best achievable error by the hypothesis class), agnostic stability is fully characterized by the Littlestone dimension. Consequently, as in the realizable case, this form of learnability is equivalent to online learnability. As part of the proof of this theorem, we strengthen the celebrated result of Bun et al. by showing that classes with infinite Littlestone dimension are not stably PAC learnable, even if we allow the stability parameter to depend on the excess error. For the second relaxation proposed by Chase et al., we prove that only finite hypothesis classes are globally stable learnable, even if we restrict the agnostic setting to distributions with small population loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。