提出新框架,统一分析朗之万采样在不同条件下的KL散度误差。
Shifted Composition III: Local Error Framework for KL Divergence
- 结合局部误差与吉尔桑诺夫定理,用简单假设获得紧致界。
- 在强/弱对数凹和LSI条件下,首次实现非Wasserstein度量的KL保证。
- 适用于多种采样算法,尤其在高维情形下达到最优速率。
耦合论证是控制两个随机过程偏差的核心工具,但传统上仅限于Wasserstein度量。本文将先前工作中提出的移位复合规则——一种信息论原理——应用于KL散度,使耦合论证可拓展至该度量。我们的框架融合了局部误差分析与吉尔桑诺夫定理的优势:既像前者一样通过引入弱误差获得紧致界,且只需易验证的局部假设;又如后者般提供KL散度保证,并超越Wasserstein收缩性限制。我们将该框架用于从目标分布π中采样,其中两个随机过程为朗之万扩散及其算法离散化。该框架在π为强对数凹(SLC)、弱对数凹(WLC)或满足对数索博列夫不等式(LSI)时,提供统一分析。结果包括对朗之万扩散的随机中点离散化的KL保证。特别地,本结果:(1) 在SLC与LSI设置下实现最优$ ilde O(\ ext{\sqrt{d}/ε})$率;(2) 是首个在SLC设置下超越2-Wasserstein度量的结果;(3) 是首个在WLC与LSI设置下任意度量下的结果。
原文摘要 · Abstract (English)
Coupling arguments are a central tool for bounding the deviation between two stochastic processes, but traditionally have been limited to Wasserstein metrics. In this paper, we apply the shifted composition rule--an information-theoretic principle introduced in our earlier work--in order to adapt coupling arguments to the Kullback-Leibler (KL) divergence. Our framework combine the strengths of two previously disparate approaches: local error analysis and Girsanov's theorem. Akin to the former, it yields tight bounds by incorporating the so-called weak error, and is user-friendly in that it only requires easily verified local assumptions; and akin to the latter, it yields KL divergence guarantees and applies beyond Wasserstein contractivity. We apply this framework to the problem of sampling from a target distribution $π$. Here, the two stochastic processes are the Langevin diffusion and an algorithmic discretization thereof. Our framework provides a unified analysis when $π$ is assumed to be strongly log-concave (SLC), weakly log-concave (WLC), or to satisfy a log-Sobolev inequality (LSI). Among other results, this yields KL guarantees for the randomized midpoint discretization of the Langevin diffusion. Notably, our result: (1) yields the optimal $\tilde O(\sqrt d/ε)$ rate in the SLC and LSI settings; (2) is the first result to hold beyond the 2-Wasserstein metric in the SLC setting; and (3) is the first result to hold in \emph{any} metric in the WLC and LSI settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。