arXiv:2510.05489cs.LGmath.OC2025-10

提出一种无需迭代的精确优化方法,可一步达到最优解。

The Method of Infinite Descent

  • 将训练转化为求解一阶最优性方程的解析问题
  • 在特定架构下实现单步收敛,无需梯度迭代
  • 适合追求精确解与非迭代学习的研究者

传统训练依赖小步局部迭代更新。通过截断梯度近似的方法可追溯至柯西与牛顿。本文提出无限下降法(Infinite Descent),一种半解析优化范式,将训练重述为直接求解一阶最优性条件。通过对泰勒展开进行解析重求和,该方法导出更新步长的精确代数方程。文中形式推导了无限泰勒塔的级联重求和实现,并提出用于直接求解的算法。该原理通过新提出的 AION(Analytic, Infinitely-Optimisable Network)架构验证:AION 专为满足无限下降所需的代数闭合性而设计,在简单测试中仅用一步下降即达最优。此优化器-模型组合展示了解析结构如何实现精确、非迭代收敛。无限下降可推广至任意适当闭合架构,由此定义一类新型半解析可优化模型——无穷类(Infinity Class);并讨论了其成员的充分条件,为非迭代学习开辟新路径。

原文摘要 · Abstract (English)

Training - the optimisation of complex models - is traditionally performed through small, local, iterative updates [D. E. Rumelhart, G. E. Hinton, R. J. Williams, Nature 323, 533-536 (1986)]. Approximating solutions through truncated gradients is a paradigm dating back to Cauchy [A.-L. Cauchy, Comptes Rendus Mathématique 25, 536-538 (1847)] and Newton [I. Newton, The Method of Fluxions and Infinite Series (Henry Woodfall, London, 1736)]. This work introduces the Method of Infinite Descent, a semi-analytic optimisation paradigm that reformulates training as the direct solution to the first-order optimality condition. By analytical resummation of its Taylor expansion, this method yields an exact, algebraic equation for the update step. Realisation of the infinite Taylor tower's cascading resummation is formally derived, and an exploitative algorithm for the direct solve step is proposed. This principle is demonstrated with the herein-introduced AION (Analytic, Infinitely-Optimisable Network) architecture. AION is a model designed expressly to satisfy the algebraic closure required by Infinite Descent. In a simple test problem, AION reaches the optimum in a single descent step. Together, this optimiser-model pair exemplify how analytic structure enables exact, non-iterative convergence. Infinite Descent extends beyond this example, applying to any appropriately closed architecture. This suggests a new class of semi-analytically optimisable models: the \emph{Infinity Class}; sufficient conditions for class membership are discussed. This offers a pathway toward non-iterative learning.

优化方法非迭代解析训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。