用严格数学方法证明一类优化算法的动态行为规律。
Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
- 提出严谨的动态空腔法，统一分析梯度下降等算法
- 首次严格推导出广义一阶算法的宏观动态方程
- 为机器学习算法的理论分析提供新工具，适合研究者参考
动力学平均场理论（DMFT）为某些无序系统中宏观可观测量的动力学提供了渐近描述。该理论最初由Sompolinsky和Zippelius（1982）在自旋玻璃背景下提出，随后被广泛应用于物理、高维统计和机器学习中的多种模型。物理学家常用的一种核心工具是动态空腔法，但其长期缺乏严格的数学基础。相比之下，现有的数学形式化多依赖于高斯条件、路径的大偏差或傅里叶分析等替代方法。本文首次形式化了动态空腔法，并利用它为广义一阶算法（General First Order Methods）给出了新的严格证明，该类算法涵盖梯度下降、近似消息传递等常见优化方法。
原文摘要 · Abstract (English)
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context of spin glasses by Sompolinsky and Zippelius (1982), it has since been used to derive asymptotic dynamical equations for a wide range of models in physics, high-dimensional statistics and machine learning. One of the main tools used by physicists to obtain these equations is the dynamical cavity method, which has remained largely non-rigorous. In contrast, existing mathematical formalizations have relied on alternative approaches, including Gaussian conditioning, large deviations over paths, or Fourier analysis. In this work, we formalize the dynamical cavity method and use it to give a new proof of the DMFT equations for General First Order Methods, a broad class of dynamics encompassing algorithms such as Gradient Descent and Approximate Message Passing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。