arXiv:2501.02298stat.MLcs.LG2025-01ICML被引 31

放松假设,提升生成模型在W2距离下的收敛性分析。

Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distance

  • 基于奥恩斯坦-乌伦贝克过程的正则化特性,弱对数凹性随时间演变为对数凹性。
  • 通过偏微分方程分析,量化了前向过程对数密度的演化规律。
  • 适用于高斯混合模型等复杂分布,适合理论研究者和模型开发者参考。

基于得分的生成模型(SGMs)通过学习受高斯噪声扰动样本的得分函数,实现从目标分布采样。现有在W2距离下的收敛性分析依赖于数据分布的严格假设,如对数凹性和得分正则性。本文提出一种新框架,显著放宽这些条件。利用奥恩斯坦-乌伦贝克(OU)过程的正则化性质,证明数据分布的弱对数凹性会随时间演化为对数凹性。这一转变通过控制前向过程对数密度的哈密顿-雅可比-贝尔曼方程进行严格量化。同时,反向时间的OU过程漂移呈现收缩与非收缩交替的动态行为,反映凹性变化。该方法避免了对得分函数及其估计器的强正则性要求,仅依赖更温和、更实用的假设。通过高斯混合模型的显式计算,验证了该框架的广泛适用性,展现出对更广类数据分布的潜力。

原文摘要 · Abstract (English)

Score-based Generative Models (SGMs) aim to sample from a target distribution by learning score functions using samples perturbed by Gaussian noise. Existing convergence bounds for SGMs in the W2-distance rely on stringent assumptions about the data distribution. In this work, we present a novel framework for analyzing W2-convergence in SGMs, significantly relaxing traditional assumptions such as log-concavity and score regularity. Leveraging the regularization properties of the Ornstein--Uhlenbeck (OU) process, we show that weak log-concavity of the data distribution evolves into log-concavity over time. This transition is rigorously quantified through a PDE-based analysis of the Hamilton--Jacobi--Bellman equation governing the log-density of the forward process. Moreover, we establish that the drift of the time-reversed OU process alternates between contractive and non-contractive regimes, reflecting the dynamics of concavity. Our approach circumvents the need for stringent regularity conditions on the score function and its estimators, relying instead on milder, more practical assumptions. We demonstrate the wide applicability of this framework through explicit computations on Gaussian mixture models, illustrating its versatility and potential for broader classes of data distributions.

生成模型理论分析概率距离得分网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。