arXiv:2508.11365cs.LGcs.AI2025-08被引 3

用代理损失优化可微规划,显著提升决策学习效果与训练速度。

Minimizing Surrogate Losses for Decision-Focused Learning using Differentiable Optimization

  • 通过最小化代理损失替代直接优化后悔值,克服线性规划梯度消失问题。
  • 在多种方法上实现与顶尖模型相当的决策质量,且训练效率大幅提升。
  • 特别适用于DYS-Net等高效可微规划框架,适合需快速训练的工业场景。

决策聚焦学习(DFL)通过训练机器学习模型预测优化问题参数,以直接最小化决策遗憾(即最大化决策质量)。基于梯度的DFL需要计算优化解对预测参数的导数,但对许多问题(如线性规划,LP)而言,后悔值对参数的梯度几乎处处为零。现有方法通常通过添加二次正则项将LP平滑为可微问题,或最小化具有信息性梯度的代理损失。本文指出,即使平滑后,后悔值在参数空间的大片区域仍为常数,导致梯度依然为零。为此,我们提出即使使用可微优化层直接最小化后悔值,也应优先最小化代理损失。实验表明,该策略使可微优化层达到甚至优于传统代理损失方法的后悔值。进一步验证了其在近期提出的DYS-Net上的有效性——该方法通过前馈神经网络层实现近似求解与梯度计算,执行正向与反向传播极为高效。利用DYS-Net最小化代理损失,可在保持与当前最优水平相当的决策质量的同时,大幅降低训练时间。

原文摘要 · Abstract (English)

Decision-focused learning (DFL) trains a machine learning (ML) model to predict parameters of an optimization problem, to directly minimize decision regret, i.e., maximize decision quality. Gradient-based DFL requires computing the derivative of the solution to the optimization problem with respect to the predicted parameters. However, for many optimization problems, such as linear programs (LPs), the gradient of the regret with respect to the predicted parameters is zero almost everywhere. Existing gradient-based DFL approaches for LPs try to circumvent this issue in one of two ways: (a) smoothing the LP into a differentiable optimization problem by adding a quadratic regularizer and then minimizing the regret directly or (b) minimizing surrogate losses that have informative (sub)gradients. In this paper, we show that the former approach still results in zero gradients, because even after smoothing the regret remains constant across large regions of the parameter space. To address this, we propose minimizing surrogate losses -- even when a differentiable optimization layer is used and regret can be minimized directly. Our experiments demonstrate that minimizing surrogate losses allows differentiable optimization layers to achieve regret comparable to or better than surrogate-loss based DFL methods. Further, we demonstrate that this also holds for DYS-Net, a recently proposed differentiable optimization technique for LPs, that computes approximate solutions and gradients through operations that can be performed using feedforward neural network layers. Because DYS-Net executes the forward and the backward pass very efficiently, by minimizing surrogate losses using DYS-Net, we are able to attain regret on par with the state-of-the-art while reducing training time by a significant margin.

决策学习可微优化代理损失线性规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。