用加速梯度法提升最大边际似然估计的收敛速度。
Momentum SVGD-EM for Accelerated Maximum Marginal Likelihood Estimation
- 在参数和概率分布空间中引入Nesterov加速，改进了粒子算法。
- 在多类任务中显著减少迭代次数，高低维场景均有效。
- 适合需要快速收敛的复杂模型参数估计任务。
最大边际似然估计（MMLE）可被看作是自由能泛函的优化问题。从这一视角出发，期望最大化（EM）算法可自然解释为在模型参数与概率测度联合空间上的坐标下降法。近年来，众多研究基于此视角提出了用于MMLE的相互作用粒子算法。本文提出一种基于斯坦因变分梯度下降（SVGD）的加速版本，通过在参数更新及概率测度空间中引入Nesterov加速，得到新方法Momentum SVGD-EM。该方法在多种难度递增的任务中均表现出更优的收敛性，在低维与高维设置下均实现迭代次数显著减少，验证了其有效性。
原文摘要 · Abstract (English)
Maximum marginal likelihood estimation (MMLE) can be formulated as the optimization of a free energy functional. From this viewpoint, the Expectation-Maximisation (EM) algorithm admits a natural interpretation as a coordinate descent method over the joint space of model parameters and probability measures. Recently, a significant body of work has adopted this perspective, leading to interacting particle algorithms for MMLE. In this paper, we propose an accelerated version of one such procedure, based on Stein variational gradient descent (SVGD), by introducing Nesterov acceleration in both the parameter updates and in the space of probability measures. The resulting method, termed Momentum SVGD-EM, consistently accelerates convergence in terms of required iterations across various tasks of increasing difficulty, demonstrating effectiveness in both low- and high-dimensional settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。