用神经网络自动调参，提升非光滑优化求解效率
Bundle Network: a Machine Learning-Based Bundle Method
- 用带注意力机制的循环神经网络替代传统梯度组合求方向
- 在多商品网络设计等任务上优于网格搜索调参方法
- 可端到端训练，跨数据集泛化能力强
本文提出基于学习的束方法——束网络（Bundle Network），用于求解凸非光滑最小化问题。与依赖启发式调参的传统方法不同，该方法从数据中自动学习调节正则化参数。此外，我们用带有注意力机制的递归神经网络替代了传统中通过访问点梯度的凸组合来计算搜索方向的迭代过程。通过利用计算图的展开结构，束网络可通过自动微分实现端到端训练。在多商品网络设计和广义分配问题的拉格朗日对偶松弛实验中，该方法在多个数据集上均显著优于依赖网格搜索调参的传统方法，且具备良好的泛化能力。
原文摘要 · Abstract (English)
This paper presents Bundle Network, a learning-based algorithm inspired by the Bundle Method for convex non-smooth minimization problems. Unlike classical approaches that rely on heuristic tuning of a regularization parameter, our method automatically learns to adjust it from data. Furthermore, we replace the iterative resolution of the optimization problem that provides the search direction-traditionally computed as a convex combination of gradients at visited points-with a recurrent neural model equipped with an attention mechanism. By leveraging the unrolled graph of computation, our Bundle Network can be trained end-to-end via automatic differentiation. Experiments on Lagrangian dual relaxations of the Multi-Commodity Network Design and Generalized Assignment problems demonstrate that our approach consistently outperforms traditional methods relying on grid search for parameter tuning, while generalizing effectively across datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。