用鲁棒优化学习最优参数，兼顾性能与泛化能力。
Distributionally-Robust Learning to Optimize
- 在凸优化中通过分布鲁棒性学习算法参数，统一了经典与最坏情况设计。
- 在多个基准上实现强泛化性能，优于最坏情况和普通学习优化方法。
- 适合追求稳健性和可证明性能保障的优化算法研究者。
我们提出一种分布鲁棒方法，用于学习一阶优化方法中的超参数。给定问题实例数据集，我们最小化一个基于Wasserstein距离的分布鲁棒性能估计问题（PEP），优化算法参数如步长。该框架统一了两个极端：当鲁棒半径趋近于零时，恢复经典学习优化（L2O）；当其增大时，恢复通过PEP得到的最坏情况最优算法设计。我们使用随机梯度下降求解，每一步对内部半定规划的解进行反向传播。我们证明了高概率界，表明所学算法的真实风险最多为样本内L2O最优值加上随样本量减小的松弛项，且不会劣于最坏情况的PEP边界。在无约束二次优化、LASSO和线性规划基准测试中，所学算法展现出优异的泛化性能，并具有可验证的鲁棒性，优于最坏情况最优和普通L2O基线。
原文摘要 · Abstract (English)
We propose a distributionally robust approach to learning hyperparameters for first-order methods in convex optimization. Given a dataset of problem instances, we minimize a Wasserstein distributionally robust version of the performance estimation problem (PEP) over algorithm parameters such as step sizes. Our framework unifies two extremes: as the robustness radius vanishes, we recover classical learning to optimize (L2O); as it grows, we recover worst-case optimal algorithm design via PEP. We solve the resulting problem with stochastic gradient descent, differentiating through the solution of an inner semidefinite program at each step. We prove high-probability bounds showing that the true risk of the learned algorithm is at most the in-sample L2O optimum plus a slack that shrinks with the sample size, and is no worse than the worst-case PEP bound. On unconstrained quadratic minimization, LASSO, and linear programming benchmarks, our learned algorithms achieve strong out-of-sample performance with certifiable robustness, outperforming both worst-case optimal and vanilla L2O baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。