统一构建去噪马尔可夫模型的数学基础,打通生成模型设计与理论分析的桥梁。
A Unified Approach to Analysis and Design of Denoising Markov Models
- 基于非平衡统计力学和广义Doob变换,建立统一建模框架
- 提出最小假设下可显式构造反向生成器和变分目标
- 适用于任意莱维型过程,支持几何布朗运动等新动态
基于测度传输的生成模型(如扩散模型、流模型)通常以马尔可夫随机动力学的形式表述,其底层过程的选择同时影响算法设计与理论分析。本文旨在为去噪马尔可夫模型这一广泛类别建立严格的数学基础:该类模型假设前向过程从目标分布演化到简单易采样的分布,同时设计特定的后向过程以实现高效的逆向采样。借助与非平衡统计力学及广义Doob's h-变换的深刻联系,我们提出一组最小假设,确保:(1) 反向生成器的显式构造;(2) 统一的变分目标,直接最小化测度传输差异;(3) 在多种动态下对经典得分匹配方法的适应。本框架统一了连续与离散扩散模型的已有形式,在前向生成器满足一定正则性条件下,揭示了最一般形式的去噪马尔可夫模型,并提供系统化方法来设计由任意莱维型过程驱动的去噪马尔可夫模型。通过引入几何布朗运动和跳跃过程作为前向动态的新型模型,展示了该框架的灵活性与实际有效性。
原文摘要 · Abstract (English)
Probabilistic generative models based on measure transport, such as diffusion and flow-based models, are often formulated in the language of Markovian stochastic dynamics, where the choice of the underlying process impacts both algorithmic design choices and theoretical analysis. In this paper, we aim to establish a rigorous mathematical foundation for denoising Markov models, a broad class of generative models that postulate a forward process transitioning from the target distribution to a simple, easy-to-sample distribution, alongside a backward process particularly constructed to enable efficient sampling in the reverse direction. Leveraging deep connections with nonequilibrium statistical mechanics and generalized Doob's $h$-transform, we propose a minimal set of assumptions that ensure: (1) explicit construction of the backward generator, (2) a unified variational objective directly minimizing the measure transport discrepancy, and (3) adaptations of the classical score-matching approach across diverse dynamics. Our framework unifies existing formulations of continuous and discrete diffusion models, identifies the most general form of denoising Markov models under certain regularity assumptions on forward generators, and provides a systematic recipe for designing denoising Markov models driven by arbitrary Lévy-type processes. We illustrate the versatility and practical effectiveness of our approach through novel denoising Markov models employing geometric Brownian motion and jump processes as forward dynamics, highlighting the framework's potential flexibility and capability in modeling complex distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。