提出新方法解决深度学习中的约束优化难题。
Stochastic Penalty-Barrier Methods for Constrained Machine Learning
- 用指数对偶平均与更鲁棒的惩罚调度处理非凸非光滑问题
- 在多达10,000个约束下，运行时间仅线性增长
- 适合需要公平性或物理一致性约束的深度学习场景
约束机器学习可实现公平性感知训练、物理信息神经网络以及将符号领域知识融入统计模型。尽管其应用重要，但目前尚无通用方法适用于深度学习中自然出现的非凸、非光滑、随机设定。我们提出随机惩罚-障碍法（SPBM），通过指数对偶平均、稳定化的惩罚调度以及Moreau包络来处理非光滑性，将经典惩罚与障碍法拓展至该设置。多个场景下的实验表明，SPBM在性能上达到或优于现有约束优化基线，且相对于无约束Adam，仅增加线性运行开销，支持最多10,000个约束。
原文摘要 · Abstract (English)
Constrained machine learning enables fairness-aware training, physics-informed neural networks, and integration of symbolic domain knowledge into statistical models. Despite its practical importance, no general method exists for the non-convex, non-smooth, stochastic setting that arises naturally in deep learning. We propose the Stochastic Penalty-Barrier Method (SPBM), which extends classical penalty and barrier methods to this setting via exponential dual averaging, a stabilized penalty schedule, and the Moreau envelope to handle non-smoothness. Experiments across multiple settings show that SPBM matches or outperforms existing constrained optimization baselines while incurring only linear runtime overhead compared to unconstrained Adam for up to 10,000 constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。