提出新框架分析离散扩散模型误差,统一理论并给出首个精确误差界。
How Discrete and Continuous Diffusion Meet: Comprehensive Analysis of Discrete Diffusion Models via a Stochastic Integral Framework
- 基于泊松积分构建离散扩散的随机积分框架
- 首次获得τ-跳跃方案在KL散度下的误差上界
- 为算法设计提供数学依据,适合研究者参考
离散扩散模型因其可处理复杂分布且采样推理高效而受到关注,但其误差分析仍不完善。本文基于勒维型随机积分,将泊松随机测度推广至时不变、状态依赖的强度形式,严格建立离散扩散模型的随机积分表述,并推导出类似伊藤积分与吉尔萨诺夫定理的变化测度定理。该框架统一并强化了现有理论结果,首次在KL散度下获得τ-跳跃方案的误差上界。通过明确误差来源,揭示了离散扩散模型的数学特性,为真实场景中高效准确算法的设计提供了指导。
原文摘要 · Abstract (English)
Discrete diffusion models have gained increasing attention for their ability to model complex distributions with tractable sampling and inference. However, the error analysis for discrete diffusion models remains less well-understood. In this work, we propose a comprehensive framework for the error analysis of discrete diffusion models based on Lévy-type stochastic integrals. By generalizing the Poisson random measure to that with a time-independent and state-dependent intensity, we rigorously establish a stochastic integral formulation of discrete diffusion models and provide the corresponding change of measure theorems that are intriguingly analogous to Itô integrals and Girsanov's theorem for their continuous counterparts. Our framework unifies and strengthens the current theoretical results on discrete diffusion models and obtains the first error bound for the $τ$-leaping scheme in KL divergence. With error sources clearly identified, our analysis gives new insight into the mathematical properties of discrete diffusion models and offers guidance for the design of efficient and accurate algorithms for real-world discrete diffusion model applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。