arXiv:2502.13747cs.LGstat.ME2025-02被引 10

通过反向马尔可夫过程分步生成复杂分布,提升图像等高难度数据建模能力。

Reverse Markov Learning: Multi-Step Generative Models for Complex Distributions

  • 构建从目标分布到已知分布的前向过程,再用多个评分模型学习反向生成路径。
  • 在气候与模拟数据上实现复杂分布建模,误差可控且训练推理更高效。
  • 适合需要高精度生成建模的科研与工业场景,尤其擅长处理高维数据。

学习复杂分布是当代应用中的基础挑战。Shen 和 Meinshausen(2024)提出 engression,一种基于评分规则的生成方法,可将噪声(及协变量,若可用)直接映射到数据。尽管有效,但 engression 在处理图像等高度复杂分布时仍存在局限。本文提出反向马尔可夫学习(RML),定义一个通用前向过程,将目标分布逐步转移到已知分布(如高斯分布),并利用多个 engression 模型学习反向马尔可夫过程,逐步重构目标分布。该框架支持任意前向过程,允许降维,并自然实现生成过程离散化。在扩散模型类前向过程下,RML 提供了高效的训练与推理离散化策略。我们进一步引入交替采样方案以提升后训练性能。统计分析建立了 RML 的误差界,揭示其在估计效率和前向过程设计灵活性方面的优势。模拟与气候数据的实证结果验证了理论发现,证明 RML 在捕捉复杂分布上的有效性。

原文摘要 · Abstract (English)

Learning complex distributions is a fundamental challenge in contemporary applications. Shen and Meinshausen (2024) introduced engression, a generative approach based on scoring rules that maps noise (and covariates, if available) directly to data. While effective, engression can struggle with highly complex distributions, such as those encountered in image data. In this work, we propose reverse Markov learning (RML), a framework that defines a general forward process transitioning from the target distribution to a known distribution (e.g., Gaussian) and then learns a reverse Markov process using multiple engression models. This reverse process reconstructs the target distribution step by step. This framework accommodates general forward processes, allows for dimension reduction, and naturally discretizes the generative process. In the special case of diffusion-based forward processes, RML provides an efficient discretization strategy for both training and inference in diffusion models. We further introduce an alternating sampling scheme to enhance post-training performance. Our statistical analysis establishes error bounds for RML and elucidates its advantages in estimation efficiency and flexibility in forward process design. Empirical results on simulated and climate data corroborate the theoretical findings, demonstrating the effectiveness of RML in capturing complex distributions.

生成模型扩散模型概率建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。