arXiv:2410.19844math.OCcs.LG2024-10

用非凸优化加速超大多项式求和平方问题,速度比传统方法快得多。

A practical, fast method for solving sum-of-squares problems for very large polynomials

  • 将凸的SDP问题转为非凸、无约束的过参数化形式,用梯度下降求解。
  • 可处理含四百万系数的超大多项式,运行时间接近线性增长。
  • 实验显示总能收敛到全局最优解,适合大规模复杂优化任务。

求和平方(SOS)优化是强制多项式非负性的有力工具。常规方法是将其松弛为半定规划(SDP),虽为凸问题且有高效求解器,但现有方法在规模上仍受限。本文提出一种受多项式神经网络启发的新方法:放弃凸的SDP形式,改用非凸、无约束、过参数化的形式,并用一阶优化方法求解。该方法可处理超过四百万系数的大型多项式,远超现有SDP方法的能力。实验表明,该方法在一般非稀疏多项式上始终收敛至正确全局最小值,运行时间仅略高于系数数量的线性增长,而传统方法则高于二次方增长。理论与实践结果均支持其避免伪局部极小值,适合基于梯度下降的快速求解。

原文摘要 · Abstract (English)

Sum of squares (SOS) optimization is a powerful technique for solving problems where the positivity of a polynomials must be enforced. The common approach to solve an SOS problem is by relaxation to a Semidefinite Program (SDP). The main advantage of this transormation is that SDP is a convex problem for which efficient solvers are readily available. However, while considerable progress has been made in recent years, the standard approaches for solving SDPs are still known to scale poorly. Our goal is to devise an approach that can handle larger, more complex problems than is currently possible. The challenge indeed lies in how SDPs are commonly solved. State-Of-The-Art approaches rely on the interior point method, which requires the factorization of large matrices. We instead propose an approach inspired by polynomial neural networks, which exhibit excellent performance when optimized using techniques from the deep learning toolbox. In a somewhat counter-intuitive manner, we replace the convex SDP formulation with a non-convex, unconstrained, and \emph{over parameterized} formulation, and solve it using a first order optimization method. It turns out that this approach can handle very large problems, with polynomials having over four million coefficients, well beyond the range of current SDP-based approaches. Furthermore, we highlight theoretical and practical results supporting the experimental success of our approach in avoiding spurious local minima, which makes it amenable to simple and fast solutions based on gradient descent. In all the experiments, our approach had always converged to a correct global minimum, on general (non-sparse) polynomials, with running time only slightly higher than linear in the number of polynomial coefficients, compared to higher than quadratic in the number of coefficients for SDP-based methods.

优化算法多项式优化非凸优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。