提出GADD方法,让离散扩散模型采样更快更准
From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models

- 用吉布斯后验直接构造校正器,无需额外训练
- 采样复杂度达O(polylog(ε⁻¹)),首次实现此速率
- 适合追求高效高质量生成的开发者与研究者
离散扩散模型在文本等符号领域表现优异,但均匀速率模型常需大量步骤生成单个样本。现有加速方法或依赖额外训练,或存在混合缓慢问题。本文提出基于吉布斯的校正器GADD,利用具体得分函数结构直接构建吉布斯后验概率,仅需标准得分估计,无需额外训练。理论证明GADD实现总体采样复杂度O(polylog(ε⁻¹)),为首个此类速率的扩散采样器。数值实验显示,在合成数据、零样本文本生成和零样本条件音乐生成任务中,GADD均显著提升样本质量和实际效率,优于标准欧拉方法与CTMC校正器。此外,理论分析提出新框架,以归纳法追踪预测迭代中的误差传播,并考虑校正更新不准确的影响,不同于传统吉尔萨诺夫变换方法。
原文摘要 · Abstract (English)
Discrete diffusion models have achieved strong empirical performance in text and other symbolic domains, but, especially for uniform-rate models, they often require many steps to generate a single sample. Existing acceleration methods either rely on training additional quantities or suffer from slow mixing. In this work, we propose a novel Gibbs-based corrector for discrete diffusion models, termed Gibbs-Accelerated Discrete Diffusion (GADD). GADD leverages the structure of the concrete score function to construct Gibbs posterior likelihoods directly, without requiring any additional training beyond standard score estimation. We show that GADD achieves an overall sampling complexity of $\mathcal{O}(\mathrm{polylog} (\varepsilon^{-1}))$, yielding the first such rate for diffusion-based samplers for uniform-rate discrete diffusion models. We also conduct numerical experiments demonstrating the practical advantages of GADD across synthetic data, zero-shot text sampling, and zero-shot conditional music generation. These results corroborate the theory and show that GADD consistently improves sample quality and wall-clock efficiency over standard baselines, including vanilla Euler methods and CTMC correctors. Beyond this, our theoretical analysis introduces a novel framework for analyzing predictor-corrector methods in discrete diffusion models, which may be of independent interest. Unlike existing approaches that rely on the Girsanov change-of-measure technique, our method is based on an induction argument that tracks error propagation across predictor iterations while accounting for inaccuracies in the corrector updates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。