用物理定律解释优化算法，发现其背后隐藏的自然运动规律。
On The Mathematics of the Natural Physics of Optimization
- 将优化问题转化为控制理论中的极小化原理，建立数学对应关系。
- 通过庞特里亚金最小值原理实现全局最优解，基于哈密顿-雅可比不等式。
- 提出反向最优算法，通过能量耗散机制自动生成多种优化方法。
许多优化算法受牛顿力学启发。本文探讨：优化算法自身是否遵循某种‘自然运动定律’？能否通过这些定律推导出算法？我们提出假设：优化算法是某些隐藏算法原语的表现，其遵循普遍的非牛顿动力学。该自然优化物理学通过将最优控制问题的终端横截条件与优化问题的广义Karush-John-Kuhn-Tucker条件等价，使给定约束优化问题的数据函数生成一个贯穿隐空间的自然向量场，传递最优性信息。通过庞特里亚金型最小值原理实现‘远距离作用’，产生局部动作以达成全局结果，依据哈密顿-雅可比不等式完成。反向最优算法通过控制跳跃耗散由搜索李雅普诺夫函数定义的量子化‘能量’而生成。实例表明，大量现有算法可在此新数学物理框架下被统一生成与解释。
原文摘要 · Abstract (English)
A number of optimization algorithms have been inspired by the physics of Newtonian motion. Here, we ask the question: do algorithms themselves obey some ``natural laws of motion,'' and can they be derived by an application of these laws? We explore this question by positing the theory that optimization algorithms may be considered as some manifestation of hidden algorithm primitives that obey certain universal non-Newtonian dynamics. This natural physics of optimization is developed by equating the terminal transversality conditions of an optimal control problem to the generalized Karush/John-Kuhn-Tucker conditions of an optimization problem. Through this equivalence formulation, the data functions of a given constrained optimization problem generate a natural vector field that permeates an entire hidden space with information on the optimality conditions. An ``action-at-a-distance'' operation via a Pontryagin-type minimum principle produces a local action to deliver a globalized result by way of a Hamilton-Jacobi inequality. An inverse-optimal algorithm is generated by performing control jumps that dissipate quantized ``energy'' defined by a search Lyapunov function. Illustrative applications of the proposed theory show that a large number of algorithms can be generated and explained in terms of the new mathematical physics of optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。