arXiv:2410.10243cs.LGcs.LO2024-10被引 7

厘清统计学习基础定理中的可测性要求,为神经网络等模型提供严格理论支撑。

Measurability in the Fundamental Theorem of Statistical Learning

  • 从测度论角度重新审视学习理论,明确证明所需的最小可测性条件。
  • 给出自包含的严格证明,并指出在非标准设定下必须关注测度论细节。
  • 适用于研究模型论、神经网络等领域的理论工作者,尤其关注结构化数据建模者。

统计学习基础定理指出,假设空间在PAC意义下可学习当且仅当其VC维有限。现有对误分类设置下该定理的证明往往隐含多个可测性假设。本文从测度论视角严格分析这些证明,显式提取出使论证成立所需的必要条件,从而给出在误分类设定下该定理的严谨表述与完整自洽证明,揭示了最弱的可测性要求。由于该定理支撑大量后续理论发展,我们的工作具有基础意义:在涉及测度论细微之处的场景中,必须审慎处理可测性问题。特别地,我们讨论了其在模型论中的应用,考虑NIP与o-极小结构。主定理给出了定义在实数o-极小扩张上的假设空间的PAC可学习性充分条件,涵盖使用ReLU、Sigmoid等常见激活函数的二分类人工神经网络。

原文摘要 · Abstract (English)

The Fundamental Theorem of Statistical Learning states that a hypothesis space is PAC learnable if and only if its VC dimension is finite. For the agnostic model of PAC learning, the literature so far presents proofs of this theorem that often tacitly impose several measurability assumptions on the involved sets and functions. We scrutinize these proofs from a measure-theoretic perspective in order to explicitly extract the assumptions needed for a rigorous argument. This leads to a sound statement as well as a detailed and self-contained proof of the Fundamental Theorem of Statistical Learning in the agnostic setting, showcasing the minimal measurability requirements needed. As the Fundamental Theorem of Statistical Learning underpins a wide range of further theoretical developments, our results are of foundational importance: A careful analysis of measurability aspects is essential, especially when the theorem is used in settings where measure-theoretic subtleties play a role. We particularly discuss applications in Model Theory, considering NIP and o-minimal structures. Our main theorem presents sufficient conditions for the PAC learnability of hypothesis spaces defined over o-minimal expansions of the reals. This class of hypothesis spaces covers all artificial neural networks for binary classification that use commonly employed activation functions like ReLU and the sigmoid function.

学习理论可测性模型论神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。