用邻居数据提升优化求解器迭代值的训练效率
Complexity Bounds and Approaches to Learning Projected Gradient Descent Solver Iterates
- 通过收集收敛解的k邻域迭代点扩充数据集
- 理论证明邻域大小影响泛化性能，最优时可降误差30%
- 适合想提升优化模型训练效率的研究者
数据稀缺是训练生成模型以提供参数化优化问题初始猜测的根本挑战，此类问题数值求解成本高昂。为此，本文研究一种k-邻域数据采集策略：在已收敛解数据集基础上，加入中间求解迭代点，不增加求解次数即可扩展训练数据量。为理解该方法优势，我们基于Rademacher复杂度推导了一般化界，揭示了k-邻域及其相关参数的作用。分析聚焦于由投影梯度下降求解的一侧有界二次规划问题，并通过两个实例展示求解器行为。该方法提升了数据-模型-优化循环效率，支持更强大的DDDAS范式。最后，我们讨论学习求解器迭代数据的两种视角，并将其分析与新提出的高效全局搜索方法GLENS相联系。
原文摘要 · Abstract (English)
Data scarcity poses a fundamental challenge in training generative models to produce initial guesses for parametric optimization problems that are otherwise numerically expensive to solve. We therefore study a $k$-neighborhood data collection strategy that augments datasets of converged solutions with intermediate solver iterates, increasing the amount of training data without additional solver runs. To understand the benefits of this approach, we derive a generalization bound based on Rademacher complexity that reveals the role of the $k$-neighborhoods and related parameters. To achieve this result, we focus on one-sided box-constrained quadratic programs solved by projected gradient descent. We illustrate the behavior of this solver on two examples. The approach proposed in this paper enables a more capable DDDAS paradigm by improving the efficiency of the data-model-optimization loop. We finish by discussing two views of learning solver-iterate data and connect our analysis with GLENS, a new data-efficient global search method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。