证明几何退火在朗之万动力学中可能无效甚至有害
Provable Convergence and Limitations of Geometric Tempering for Langevin Dynamics
- 从函数不等式角度分析几何退火的收敛性
- 发现某些情况下收敛时间呈指数级增长
- 揭示即使目标分布良好也存在收敛瓶颈
几何退火是一种通过在简单先验分布与目标分布之间使用几何平均构建一系列中间分布来采样复杂多模态分布的方法。本文针对朗之万动力学采样算法,首次在函数不等式框架下进行理论分析,证明了连续与离散时间下的收敛上界;其最小化可导出特定分布对下的闭式最优退火路径。同时,下界分析表明,在简单案例中几何退火需指数时间收敛,并揭示其可能因功能不等式差而导致缓慢收敛,即使目标分布条件良好。总体表明,几何退火未必有效,甚至可能阻碍收敛。
原文摘要 · Abstract (English)
Geometric tempering is a popular approach to sampling from challenging multi-modal probability distributions by instead sampling from a sequence of distributions which interpolate, using the geometric mean, between an easier proposal distribution and the target distribution. In this paper, we theoretically investigate the soundness of this approach when the sampling algorithm is Langevin dynamics, proving both upper and lower bounds. Our upper bounds are the first analysis in the literature under functional inequalities. They assert the convergence of tempered Langevin in continuous and discrete-time, and their minimization leads to closed-form optimal tempering schedules for some pairs of proposal and target distributions. Our lower bounds demonstrate a simple case where the geometric tempering takes exponential time, and further reveal that the geometric tempering can suffer from poor functional inequalities and slow convergence, even when the target distribution is well-conditioned. Overall, our results indicate that geometric tempering may not help, and can even be harmful for convergence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。