深度学习中应优先用约束而非固定惩罚，更可靠地满足硬性要求。
Position: Adopt Constraints Over Fixed Penalties in Deep Learning
- 用约束直接建模非协商性要求，避免软化为可调惩罚。
- 固定惩罚在非凸问题中无法保证等价性，可能解错问题。
- 适合需要严格遵守规则的场景，如医疗、自动驾驶等可信AI应用。
近期对可信人工智能系统的研究推动了对显式需求（约束）的学习问题关注。然而，在深度学习中，这类问题常通过固定加权求和惩罚处理：将约束加入任务损失并用固定系数加权，再最小化得到的标量目标。本文认为，固定惩罚在具有不可协商要求的深度学习问题中通常不适用。首先，在非凸设置下，惩罚问题与约束问题一般不等价，求解前者未必能解决后者。其次，固定惩罚将硬性要求弱化为可与任务性能权衡的软项。第三，为间接求解约束问题而调整惩罚系数往往需耗时试错，因改变系数会改变惩罚目标本身，可能导致完全解错问题。因此，当深度学习问题包含不可协商的要求时，应以约束形式为起点，而非固定惩罚的代理问题。解决方案的选择应基于问题的结构和规模。
原文摘要 · Abstract (English)
Recent efforts to develop trustworthy AI systems have increased interest in learning problems with explicit requirements, or constraints. In deep learning, however, such problems are often handled through fixed weighted-sum penalization: the constraints are added to the task loss with fixed coefficients, and the resulting scalarized objective is minimized. This position paper argues that fixed penalization is often ill-suited for deep learning problems with non-negotiable requirements for several reasons. First, in non-convex settings, the penalized and constrained problems are generally not equivalent, so solving the former need not solve the latter. Second, fixed penalization weakens hard requirements into soft penalties to be traded off against task performance. Third, choosing penalty coefficients to indirectly solve the constrained problem often involves costly trial and error, because changing them alters the penalized objective itself, and hence can mean solving the wrong problem altogether. We therefore argue that, when a deep learning problem specifies non-negotiable requirements, the constrained formulation itself should be the starting point, not the surrogate problem defined by fixed penalization. The appropriate solution strategy should then be chosen based on the problem's structure and scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。