arXiv:2607.06723math.OCcs.LG2026-07被引 1

提出优化几何动力学框架,用微分几何解析自适应优化器的隐藏状态演化。

Optimization Geometrodynamics: Variational Reduction and Interaction Curvature

  • 基于变分法构建优化器隐藏状态的几何理论,实现跨层级组合与消隐。
  • 导出隐式敏感度与舒尔补曲率,揭示参数更新的内在机制。
  • 提供可验证的收敛性保证与后验停止准则,适合高精度优化研究者。

自适应优化器包含随时间演化的隐藏状态,影响梯度如何转化为参数运动。本文建立优化几何动力学作为该隐藏几何的变分理论。通过极小前推消除等效作用的隐藏状态,并支持优化器层次间的组合。在光滑非退化条件下,导出隐式敏感度与松弛后的舒尔补曲率。对仿射预约扰动,诱导的交互曲率为负半定算子 $-G^*H^{-1}G$,其混合项积分产生有限机制对比。核心成果是行列式为一的仿射不变正定对称动作映射 $P\mapsto PA$。证明存在全局解析丛,纤维为闭合全测地线,且有唯一解析最近控制器截面。强凸纤维定理与显式对数动作残差,确保从任意可行初值出发的全局线性收敛求解器,附带非渐近值、控制器距离与残差界,以及可观测的后验停止证书。条件性不精确结果可传播严格残差误差与半径上界。密集谱核被限制在维度 $r\le 2m$ 的活跃子空间内,给出显式谱-算术运算界,当 $r<d$ 时实现严格降维。对嵌套形状归一化二次动作,标准多切线投影满足精确 CAT(0) 毕达哥拉斯下降,并在尖锐秩阈值 $d-1$ 处恢复行列式为一的逆海森矩阵形状,前提是标量规范 $c_H=(\det H)^{1/d}$ 已知。这些结果使动作丛成为精确迭代计算系统,具备后验证书与有限识别理论。

原文摘要 · Abstract (English)

Adaptive optimizers carry hidden states that change how visible gradients become parameter motion. We develop optimization geometrodynamics as a variational theory of this hidden geometry. Infimal pushforward eliminates all hidden states realizing the same action and composes across optimizer hierarchies. Under smooth nondegeneracy, it yields hidden susceptibility and the Schur-complement curvature seen after relaxation. For affine pre-reduction perturbations, the induced interaction curvature is the negative-semidefinite operator $-G^*H^{-1}G$, whose mixed entries integrate to finite mechanism contrasts. Our main realization is the determinant-one affine-invariant SPD action map $P\mapsto PA$. We prove a global analytic bundle with closed totally geodesic fibers and a unique analytic nearest-controller section. A strongly convex fiber theorem and an explicit logarithmic action residual give a globally linearly convergent solver from every feasible initializer, together with nonasymptotic value, controller-distance, and residual bounds and observable posterior stopping certificates. A conditional inexact result propagates supplied rigorous residual-error and radius majorants. The dense spectral kernel is confined to an active subspace of dimension $r\le 2m$, yielding an explicit spectral-arithmetic operation bound and a strict dimensional reduction when $r<d$. For nested shape-normalized quadratic actions, canonical multi-secant projections satisfy an exact CAT(0) Pythagorean decrease and recover the determinant-one inverse Hessian shape at the sharp rank threshold $d-1$, provided the scalar gauge $c_H=(\det H)^{1/d}$ is known. These results turn the action bundle into an exact iterative computation with posterior certificates and a finite-identification theory.

优化理论几何动力学收敛分析数学优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。