KL散度是唯一无需归一化常数即可用梯度流采样的发散度。
A note on the unique properties of the Kullback--Leibler divergence for sampling via gradient flows
- 使用梯度流优化时，仅KL散度无需知道目标分布的归一化常数。
- 在多种常见度量下，该性质使采样计算更高效可靠。
- 适合从事概率推断与生成模型的研究者阅读。
我们研究从具有密度的概率分布 π 采样的问题，该问题可转化为在概率分布空间中最小化与 π 之间的发散度的优化问题。通常通过概率分布空间中的梯度流求解，需选择合适的度量。本文证明：在Bregman发散族中，唯有KL散度在多种常见度量下，其梯度流无需依赖 π 的归一化常数，从而避免了对难以计算的归一化常数的依赖，提升了采样效率与可行性。
原文摘要 · Abstract (English)
We consider the problem of sampling from a probability distribution $π$ which admits a density w.r.t. a dominating measure. It is well known that this can be written as an optimisation problem over the space of probability distributions in which we aim to minimise a divergence from $π$. The optimisation problem is normally solved through gradient flows in the space of probability distributions with an appropriate metric. We show that the Kullback--Leibler divergence is the only divergence in the family of Bregman divergences whose gradient flow w.r.t. many popular metrics does not require knowledge of the normalising constant of $π$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。