证明生成模型的ELBO在驻点处等于熵和，揭示其理论本质。
Generative Models with ELBOs Converging to Entropy Sums
- 通过验证定理条件，推导出多种生成模型在驻点处的ELBO等于熵和。
- 在有限数据、模型不匹配等现实条件下，该等式仍成立。
- 适用于主成分分析、混合模型等经典生成模型，适合理论研究者。
证据下界（ELBO）是概率无监督学习中最核心的目标函数之一。本文证明了若干生成模型及其模型类别的ELBO收敛于熵和。作为结果，我们列出了迄今为止已证明熵收敛的生成模型，并给出相应的熵和表达式。涵盖诸如概率主成分分析、逻辑信念网络或高斯混合模型等著名模型，还涉及更广泛的指数族分布混合模型类。主要贡献在于对各模型的严格证明：针对每个模型，我们验证了[arXiv:2209.03077]中定理1或定理2的条件满足，从而根据定理可知，该模型的ELBO在所有驻点处等于熵和。该等式在现实条件下成立：有限数据点、模型与数据不匹配、任意驻点（包括鞍点）等情形下均有效，且对任意良定义的变分分布族都适用。
原文摘要 · Abstract (English)
The evidence lower bound (ELBO) is one of the most central objectives for probabilistic unsupervised learning. For the ELBOs of several generative models and model classes, we here prove convergence to entropy sums. As one result, we provide a list of generative models for which entropy convergence has been shown, so far, along with the corresponding expressions for entropy sums. Our considerations include very prominent generative models such as probabilistic PCA, sigmoid belief nets or Gaussian mixture models. However, we treat more models and entire model classes such as general mixtures of exponential family distributions. Our main contributions are the proofs for the individual models. For each given model we show that the conditions stated in Theorem 1 or Theorem 2 of [arXiv:2209.03077] are fulfilled such that by virtue of the theorems the given model's ELBO is equal to an entropy sum at all stationary points. The equality of the ELBO at stationary points applies under realistic conditions: for finite numbers of data points, for model/data mismatches, at any stationary point including saddle points etc, and it applies for any well behaved family of variational distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。