通过高斯比较定理，建立模型训练动态的简化分析框架。
A Gaussian Comparison Theorem for Training Dynamics in Machine Learning
- 基于戈登比较定理，构建可解析的近似动力学系统。
- 严格证明渐近场景下动态平均场表达式的有效性。
- 适用于分析感知机等模型在非渐近情况下的波动特性。
我们研究数据服从高斯混合模型的训练算法。针对一类特定算法，提出非渐近结果，将模型演化与一个更易分析的代理动力学系统关联。该结果的证明基于著名的戈登比较定理。利用该定理，我们严格证明了在渐近情形下动态平均场（DMF）表达式的正确性。此外，我们提出一种迭代优化方案，以获得非渐近情形下的更精确表达式。我们将理论应用于通用一阶（全批量）算法训练感知机模型，发现除DMF核外，非渐近域中还出现波动参数。
原文摘要 · Abstract (English)
We study training algorithms with data following a Gaussian mixture model. For a specific family of such algorithms, we present a non-asymptotic result, connecting the evolution of the model to a surrogate dynamical system, which can be easier to analyze. The proof of our result is based on the celebrated Gordon comparison theorem. Using our theorem, we rigorously prove the validity of the dynamic mean-field (DMF) expressions in the asymptotic scenarios. Moreover, we suggest an iterative refinement scheme to obtain more accurate expressions in non-asymptotic scenarios. We specialize our theory to the analysis of training a perceptron model with a generic first-order (full-batch) algorithm and demonstrate that fluctuation parameters in a non-asymptotic domain emerge in addition to the DMF kernels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。