arXiv:2602.20585stat.MLcs.LG2026-02

揭示分布自适应对手下学习的条件,统一在线与隐私学习理论。

Characterizing Online and Private Learnability under Distributional Constraints via Generalized Smoothness

  • 引入广义平滑性概念,刻画可学习的分布族特征。
  • 证明任意有限VC维假设类在广义平滑分布下具可控后悔界。
  • 连接在线学习与差分隐私,给出私有学习的完整刻画。

理解使学习与泛化成为可能的最小假设,是学习理论的核心问题。统计学习理论中的经典结果,如VC定理和Littlestone对在线可学习性的刻画,分别建立了独立数据与对抗数据下的学习条件。本文基于近期连接两者的研究,分析在分布自适应对手下的序列决策问题:对手从固定分布族$U$中动态选择生成数据的分布。我们研究何时此类问题能以类似独立情况的样本复杂度实现学习。通过引入广义平滑性概念,我们几乎完全刻画了可学习的分布族:一个分布族允许所有有限VC维假设类获得依赖于VC维的后悔界,当且仅当该族是广义平滑的。此外,我们给出了无需事先知道$U$即可实现低后悔的通用算法。当$U$已知时,我们进一步利用组合参数碎片数(fragmentation number)提供更精细的界限,该参数衡量$U$下可承载非零质量的互不相交区域的最大数量。这些结果近乎完整地揭示了分布约束下的可学习性。此外,借助在线学习与差分隐私之间的意外联系,我们证明广义平滑性也完全刻画了分布约束下的私有学习。

原文摘要 · Abstract (English)

Understanding minimal assumptions that enable learning and generalization is perhaps the central question of learning theory. Several celebrated results in statistical learning theory, such as the VC theorem and Littlestone's characterization of online learnability, establish conditions on the hypothesis class that allow for learning under independent data and adversarial data, respectively. Building upon recent work bridging these extremes, we study sequential decision making under distributional adversaries that can adaptively choose data-generating distributions from a fixed family $U$ and ask when such problems are learnable with sample complexity that behaves like the favorable independent case. We provide a near complete characterization of families $U$ that admit learnability in terms of a notion known as generalized smoothness i.e. a distribution family admits VC-dimension-dependent regret bounds for every finite-VC hypothesis class if and only if it is generalized smooth. Further, we give universal algorithms that achieve low regret under any generalized smooth adversary without explicit knowledge of $U$. Finally, when $U$ is known, we provide refined bounds in terms of a combinatorial parameter, the fragmentation number, that captures how many disjoint regions can carry nontrivial mass under $U$. These results provide a nearly complete understanding of learnability under distributional adversaries. In addition, building upon the surprising connection between online learning and differential privacy, we show that the generalized smoothness also characterizes private learnability under distributional constraints.

学习理论在线学习差分隐私分布对抗

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。