从频率学派视角重新解读变分推断，打通统计理论与生成模型的桥梁。
A Frequentist Statistical Introduction to Variational Inference, Autoencoders, and Diffusion Models
- 用经典EM算法出发，将变分推断视为处理不可计算E步的可扩展解法。
- 揭示变分自编码器与扩散模型是频率学派下最大似然估计的深度学习延伸。
- 适合希望理解生成模型统计根基的学者，尤其关注理论衔接的统计学家。
尽管变分推断（VI）是现代生成模型如变分自编码器（VAEs）和去噪扩散模型（DDMs）的核心，其教学内容却分散在不同学科中。在统计学中，VI通常被当作贝叶斯后验近似的工具；而在机器学习中，VAEs和DDMs则从频率学派视角出发，使用VI来逼近最大似然估计器。这一差异对统计学家构成障碍，因为缺乏对应的频率学派框架难以理解VAEs和DDMs的原理。本文提供这样的引入：我们从纯频率学派视角解释VI、VAEs和DDMs的理论基础，始于经典的期望最大化（EM）算法。我们展示如何将VI作为处理不可计算的E步的可扩展解法，并说明VAEs和DDMs是该框架的自然、基于深度学习的扩展，从而弥合了经典统计推断与现代生成式人工智能之间的鸿沟。
原文摘要 · Abstract (English)
While Variational Inference (VI) is central to modern generative models like Variational Autoencoders (VAEs) and Denoising Diffusion Models (DDMs), its pedagogical treatment is split across disciplines. In statistics, VI is typically framed as a Bayesian method for posterior approximation. In machine learning, however, VAEs and DDMs are developed from a Frequentist viewpoint, where VI is used to approximate a maximum likelihood estimator. This creates a barrier for statisticians, as the principles behind VAEs and DDMs are hard to contextualize without a corresponding Frequentist introduction to VI. This paper provides that introduction: we explain the theory for VI, VAEs, and DDMs from a purely Frequentist perspective, starting with the classical Expectation-Maximization (EM) algorithm. We show how VI arises as a scalable solution for intractable E-steps and how VAEs and DDMs are natural, deep-learning-based extensions of this framework, thereby bridging the gap between classical statistical inference and modern generative AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。