arXiv:2502.10354cs.LGmath.ST2025-02NeurIPS被引 1

提出无需依赖维度的得分匹配方法,显著提升扩散模型训练效率。

Dimension-free Score Matching and Time Bootstrapping for Diffusion Models

  • 用统一函数逼近器联合估计多噪声水平下的得分函数
  • 实现近似无维度依赖的样本复杂度,比之前方法快数倍
  • 适合研究生成模型理论或想优化扩散模型训练的人

扩散模型通过在不同噪声水平下估计目标分布的得分函数来生成样本,训练过程逐步添加噪声。以往的样本复杂度界对维度 $d$ 呈多项式依赖,仅除 $/log(|/mathcal{H}|)$ 外。本文首次建立(近)无维度依赖的样本复杂度界,仅保留 $/log(|/mathcal{H}|)$ 项,相比之前结果在维度上实现双指数级改进。分析关键在于使用单一函数逼近器联合估计各噪声水平的得分,该设计可实现跨时间步的泛化。我们引入基于鞅的误差分解和紧致方差界,使模型能高效学习由马尔可夫过程生成的依赖数据,该技术本身亦具独立价值。在此基础上,我们提出自举得分匹配(Bootstrapped Score Matching, BSM),利用先前学习的得分函数提升高噪声水平下的精度。这些成果为扩散模型在生成建模中的高效性与有效性提供了理论支持。

原文摘要 · Abstract (English)

Diffusion models generate samples by estimating the score function of the target distribution at various noise levels. The model is trained using samples drawn from the target distribution by progressively adding noise. Previous sample complexity bounds have polynomial dependence on the dimension $d$, apart from a $\log(|\mathcal{H}|)$ term, where $\mathcal{H}$ is the hypothesis class. In this work, we establish the first (nearly) dimension-free sample complexity bounds, modulo the $\log(|\mathcal{H}|)$ dependence, for learning these score functions, achieving a double exponential improvement in the dimension over prior results. A key aspect of our analysis is the use of a single function approximator to jointly estimate scores across noise levels, a practical feature that enables generalization across time steps. We introduce a martingale-based error decomposition and sharp variance bounds, enabling efficient learning from dependent data generated by Markov processes, which may be of independent interest. Building on these insights, we propose Bootstrapped Score Matching (BSM), a variance reduction technique that leverages previously learned scores to improve accuracy at higher noise levels. These results provide insights into the efficiency and effectiveness of diffusion models for generative modeling.

扩散模型得分匹配理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。