提出新采样方法,用冷启动点加速多峰分布混合。
Sampling from multimodal distributions with warm starts: Non-asymptotic bounds for the Reweighted Annealed Leap-Point Sampler
- 用冷启动点构造倾斜分布,通过跳跃实现跨峰混合
- 首次证明在一般条件下多项式时间收敛,无需高斯假设
- 适合复杂几何的多峰分布,尤其重尾分布场景
从多峰分布中采样是贝叶斯推断和机器学习的核心挑战。尽管经典MCMC方法(即使采用温化)可能面临指数级混合时间,一个自然问题是:如何利用每个模式的冷启动点信息来加快跨峰混合?为此,我们提出重加权安热跳跃点采样器(Re-ALPS),该方法无需对各模式做高斯近似。我们在一个自然假设下证明了首个适用于一般设置的多项式时间收敛界:当各成分在朝向对应冷启动点倾斜时仍保持显著质量。与ALPS类似,我们定义朝向冷启动点混合中心倾斜的分布,并在最冷层级使用点间跳跃以实现高效跨峰混合。不同于ALPS,本方法不依赖模式处的海森矩阵信息,而是通过蒙特卡洛估计各成分的分区函数。这一额外估计步骤使算法可处理除近似高斯外更复杂的几何结构。理论分析中,我们给出了部分平稳分布良好混合时马尔可夫过程的收敛结果,并对混合成分的分区函数估计进行了分析。我们在重尾分布混合上数值评估了算法性能,结果表明其相比ALPS具有更优的混合表现。
原文摘要 · Abstract (English)
Sampling from multimodal distributions is a central challenge in Bayesian inference and machine learning. In light of hardness results for sampling -- classical MCMC methods, even with tempering, can suffer from exponential mixing times -- a natural question is how to leverage additional information, such as a warm start point for each mode, to enable faster mixing across modes. To address this, we introduce Reweighted ALPS (Re-ALPS), a modified version of the Annealed Leap-Point Sampler (ALPS) that dispenses with the Gaussian approximation assumption. We prove the first polynomial-time bound that works in a general setting, under a natural assumption that each component contains significant mass relative to the others when tilted towards the corresponding warm start point. Similarly to ALPS, we define distributions tilted towards a mixture centered at the warm start points, and at the coldest level, use teleportation between warm start points to enable efficient mixing across modes. In contrast to ALPS, our method does not require Hessian information at the modes, but instead estimates component partition functions via Monte Carlo. This additional estimation step is crucial in allowing the algorithm to handle target distributions with more complex geometries besides approximate Gaussian. For the proof, we show convergence results for Markov processes when only part of the stationary distribution is well-mixing and estimation for partition functions for individual components of a mixture. We numerically evaluate our algorithm's mixing performance compared to ALPS on a mixture of heavy-tailed distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。