arXiv:2608.08414cs.LGmath.OC2026-08

提出可统一学习的约束学习框架,实现最优与可行双保障。

Constrained Learning with Universally Learnable Concept Classes

  • 在无限维假设类中建立双算法的普适学习性,兼顾最优与约束满足。
  • 证明最优值可精确学习,样本阈值关于误差为多项式增长。
  • 引入闭包实现差距衡量可行性,适用于无分布假设的大模型学习。

研究完全非凸设置下无限维假设类的约束统计学习,确立了双算法解的普适PACC可学习性:即在概率近似正确意义下同时保证最优性与约束满足。该结果强化了此前近似PACC结果,其可行性残差无法通过增加数据消除。最优性受制于两个矛盾机制:由Rademacher复杂度控制的泛化(偏好小类)和基于Lyapunov凸性的强对偶性(需可分解性)。本文通过在通用RKHS $/mathcal{H}_K$ 上定义总体问题(稠密于可分解包络),并沿增长半径的范数球进行学习,导出Tikhonov复杂度 $ extit{\mathfrak{T}}^{ ext{\varepsilon}}_{n}$,即达到 $\varepsilon$-最优拉格朗日水平集的最小RKHS范数;证明其有限性,获得最优值的精确学习,并使样本阈值显式且在源条件下关于 $1/\varepsilon$ 为多项式。可行性更难:无凸性时拉格朗日可能不达下确界,且对偶信息仅给出平均约束风险向量而非任意预测器的风险。为此引入闭包实现差距 $\varepsilon^{\star}_{\infty}$,作为问题固有属性,衡量 $\mathcal{H}_K$ 从对偶化中恢复可行解的能力;当 $\varepsilon^{\star}_{\infty}=0$ 时可精确学习,尤其在对偶可微时成立;否则为近似PACC,残差恰为 $\varepsilon^{\star}_{\infty}$。最后,即使在无约束特例中也不存在分布无关的样本阈值,因此对大规模假设类的对偶算法而言,普适性是自然框架。

原文摘要 · Abstract (English)

We study constrained statistical learning over infinite-dimensional hypothesis classes in the fully nonconvex setting, and establish universal PACC learnability of the solutions of dual algorithms: Probably Approximately Correct on Constraints, guaranteeing optimality and constraint satisfaction at once. This strengthens near-PACC results, whose feasibility residual no amount of data can remove. Optimality is caught between generalization, governed by Rademacher complexity and favoring small classes, and strong Lagrangian duality, which rests on Lyapunov convexity for vector measures and needs decomposability, a demand pulling the other way. We reconcile the two by posing the population problem over a universal RKHS $\mathcal{H}_K$, dense in a decomposable envelope, and learning over norm balls of growing radius. This yields the Tikhonov complexity $\mathfrak{T}^{\varepsilon}_{n}$, the least RKHS norm reaching an $\varepsilon$-optimal Lagrangian level set; we prove it finite, obtain exact learnability of the optimal value, and make the sample threshold explicit and polynomial in $1/\varepsilon$ under a source condition. Feasibility is harder: absent convexity the Lagrangian may not attain its infimum, and dual information pins down only an averaged constraint-risk vector, not the risks of any returned predictor. We introduce the closure-realization gap $\varepsilon^\star_\infty$, an index of how well $\mathcal{H}_K$ retrieves feasible solutions from dualization; it is a property of the problem, not of a modeling choice. Learnability is exact when $\varepsilon^\star_\infty=0$, in particular under dual differentiability, and near-PACC with residual exactly $\varepsilon^\star_\infty$ otherwise. Finally, no distribution-free threshold exists already in the unconstrained specialization, so universality is the canonical frame for dual algorithms over large hypothesis classes.

约束学习泛化理论对偶优化可学习性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。