arXiv:2512.05092stat.MLcs.LG2025-12被引 2

统一连续与离散扩散模型的理论框架,揭示其核心机制。

Foundations of Diffusion Models in General State Spaces: A Self-Contained Introduction

  • 从马尔可夫核出发构建前后向扩散过程,覆盖连续与离散空间。
  • 推导出适用于各类扩散模型的通用变分下界(ELBO)。
  • 为初学者、从业者和专家提供分层递进的理论入门路径。

尽管扩散模型在生成建模中占据核心地位,但现有入门材料通常假设数据为欧几里得空间,且很少阐明其与离散状态模型的联系。本文是一篇关于一般状态空间上扩散过程的自包含引论,将连续域与离散/类别结构统一于同一理论框架下。我们并行发展离散时间视角(前向加噪使用马尔可夫核,反向由学习动力学驱动)及其连续时间极限——$ ^d$ 中的随机微分方程(SDEs)与有限字母表上的连续时间马尔可夫链(CTMCs),并推导相应的福克-普朗克方程与主方程。一种共同的变分处理导出了支撑标准训练损失的证据下界(ELBO)。本文明确展示了前向破坏方式的选择——连续空间中的高斯过程,以及离散空间中的结构化类别转移核(均匀、掩码/吸收等)——如何决定反向动态与ELBO形式。内容分层设计,服务于三类读者:寻求直观自包含入门的新手;希望获得全局理论整合的扩散模型实践者;以及希望从类比入手进入离散扩散的连续扩散专家。最终形成跨越连续域与离散序列的现代扩散方法统一路线图,突出一组可复用的证明、恒等式与核心理论原则。

原文摘要 · Abstract (English)

Although diffusion models now occupy a central place in generative modeling, introductory treatments commonly assume Euclidean data and seldom clarify their connection to discrete-state analogues. This article is a self-contained primer on diffusion over general state spaces, unifying continuous domains and discrete/categorical structures under one lens. We develop the discrete-time view (forward noising via Markov kernels and learned reverse dynamics) alongside its continuous-time limits -- stochastic differential equations (SDEs) in $\mathbb{R}^d$ and continuous-time Markov chains (CTMCs) on finite alphabets -- and derive the associated Fokker--Planck and master equations. A common variational treatment yields the ELBO that underpins standard training losses. We make explicit how forward corruption choices -- Gaussian processes in continuous spaces and structured categorical transition kernels (uniform, masking/absorbing and more) in discrete spaces -- shape reverse dynamics and the ELBO. The presentation is layered for three audiences: newcomers seeking a self-contained intuitive introduction; diffusion practitioners wanting a global theoretical synthesis; and continuous-diffusion experts looking for an analogy-first path into discrete diffusion. The result is a unified roadmap to modern diffusion methodology across continuous domains and discrete sequences, highlighting a compact set of reusable proofs, identities, and core theoretical principles.

扩散模型理论基础离散扩散统一框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。