arXiv:2604.22746math.OCcs.LG2026-04

训练神经网络时加入约束,让后续优化更快求解。

Relaxation-Informed Training of Neural Network Surrogate Models

  • 用边界和不稳定性惩罚项直接优化网络结构以提升整数规划求解效率。
  • 实验显示求解时间可缩短至原来的万分之一,且预测精度仍良好。
  • 适合需要全局优化的复杂系统建模与决策问题研究者。

ReLU神经网络作为代理模型嵌入混合整数线性规划(MILP)中可实现对学习函数的全局优化。其可求解性取决于网络结构特性,如关联公式中的二元变量数量和连续松弛的紧致性。这些特性在训练过程中决定,但标准训练目标(预测损失加经典权重正则化)无法直接控制。本文研究了针对下游MILP可求解性的训练正则化方法。提出基于边界的简单正则化项,惩罚MILP公式的big-M常数和不稳定神经元数量;同时引入松弛间隙正则化项,显式惩罚训练点处连续松弛的每样本间隙。推导其梯度并基于线性规划对偶变量实现无需自定义自动微分工具的计算。结果表明,结合多种正则化可近似捕捉松弛间隙对网络参数的全导数,涵盖直接与间接敏感性。在非凸基准函数及带分位数神经网络代理的两阶段随机规划问题上的实验表明,相比未正则化基线,所提方法可使MILP求解时间减少高达四个数量级,同时保持竞争力的代理模型精度。

原文摘要 · Abstract (English)

ReLU neural networks trained as surrogate models can be embedded exactly in mixed-integer linear programs (MILPs), enabling global optimization over the learned function. The tractability of the resulting MILP depends on structural properties of the network, i.e., the number of binary variables in associated formulations and the tightness of the continuous LP relaxation. These properties are determined during training, yet standard training objectives (prediction loss with classical weight regularization) offer no mechanism to directly control them. This work studies training regularizers that directly target downstream MILP tractability. Specifically, we propose simple bound-based regularizers that penalize the big-M constants of MILP formulations and/or the number of unstable neurons. Moreover, we introduce an LP relaxation gap regularizer that explicitly penalizes the per-sample gap of the continuous relaxation at training points. We derive its associated gradient and provide an implementation from LP dual variables without custom automatic differentiation tools. We show that combining the above regularizers can approximate the full total derivative of the LP gap with respect to the network parameters, capturing both direct and indirect sensitivities. Experiments on non-convex benchmark functions and a two-stage stochastic programming problem with quantile neural network surrogates demonstrate that the proposed regularizers can reduce MILP solve times by up to four orders of magnitude relative to an unregularized baseline, while maintaining competitive surrogate model accuracy.

神经网络优化整数规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。