arXiv:2509.16756cs.LGeess.SP2025-09NeurIPS被引 19

提出新分析方法,让离散扩散模型采样更高效可靠。

Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees

  • 用微分不等式替代传统方法,无需苛刻假设。
  • τ-跳跃采样收敛速度与词表大小线性相关,优于此前二次依赖。
  • 首次为欧拉法和Tweedie τ-跳跃提供理论保证,适合算法研究者。

离散扩散模型在自然语言和图数据应用中日益重要,其性能关键在于采样器的效率。其中,τ-跳跃采样因理论与实证表现优异而广受欢迎。然而,现有理论分析常依赖难以验证的正则性假设,且收敛界对词表大小呈二次依赖。本文提出一种新分析框架,摆脱了这些限制。针对标准τ-跳跃方法,建立了在KL散度下的收敛保证,其依赖关系与词表大小线性相关,优于先前结果。该方法更具普适性,首次为欧拉法与Tweedie τ-跳跃提供了收敛性证明。核心是基于微分不等式的新型技术,为随机过程分析提供了更灵活的替代方案,亦可能独立具有研究价值。

原文摘要 · Abstract (English)

Discrete diffusion models have recently gained significant prominence in applications involving natural language and graph data. A key factor influencing their effectiveness is the efficiency of discretized samplers. Among these, $τ$-leaping samplers have become particularly popular due to their theoretical and empirical success. However, existing theoretical analyses of $τ$-leaping often rely on somewhat restrictive and difficult-to-verify regularity assumptions, and their convergence bounds contain quadratic dependence on the vocabulary size. In this work, we introduce a new analytical approach for discrete diffusion models that removes the need for such assumptions. For the standard $τ$-leaping method, we establish convergence guarantees in KL divergence that scale linearly with vocabulary size, improving upon prior results with quadratic dependence. Our approach is also more broadly applicable: it provides the first convergence guarantees for other widely used samplers, including the Euler method and Tweedie $τ$-leaping. Central to our approach is a novel technique based on differential inequalities, offering a more flexible alternative to the traditional Girsanov change-of-measure methods. This technique may also be of independent interest for the analysis of other stochastic processes.

扩散模型采样器理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。