让AI在所有数据点上都满足损失约束，提升模型可靠性。
Everywhere Learning: Artificial Intelligence with Pointwise Constraints
- 通过双变量重加权数据分布，聚焦难满足约束的样本点。
- 实证与理论证明了经验解与统计解的接近性。
- 适合对模型鲁棒性要求高的场景，如语言模型任务。
处处学习是一种新范式，要求人工智能系统在数据分布上以概率1满足损失约束，而非传统最小化平均损失。本文发展近似对偶理论，建立经验与统计处处学习问题解之间接近性的泛化分析。结果表明，对偶变量会将数据分布重新加权，偏向于损失约束更难满足的点；泛化性能由数据分布质量集中与约束难满足点质量集中的不匹配程度控制。此外，我们发现可通过在约束松弛上施加稀疏L1正则项来调控泛化能力。实验展示了该方法在语言模型任务中代理分类的优越性。
原文摘要 · Abstract (English)
Everywhere learning is a new paradigm whereby Artificial Intelligence (AI) systems are trained to satisfy loss constraints with probability one over the data distribution. This is in contrast to the standard paradigm of training AI systems to minimize average losses. We develop an approximate duality theory to substantiate a generalization analysis that establishes the proximity between solutions of empirical and statistical everywhere learning problems. Our results show that dual variables reweigh the data distribution towards points in which loss constraints are more difficult to satisfy and that generalization is controlled by the mismatch between the concentration of mass of the data distribution and the concentration of mass on points where constraints are more difficult to satisfy. We further show that we can control generalization with a sparse L1 penalty on constraint relaxations. We illustrate the merits of everywhere learning with an experiment in agentic classification for language model tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。