提出并行时间采样算法，让离散扩散模型生成速度提升7-9倍。
Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling
- 用连续时间随机积分和皮卡迭代，实现采样并行化。
- 合成数据最快提速9倍，图像文本任务提速1.86倍且质量不变。
- 适合需要高效推理的分子生成、语言建模等场景。
离散扩散模型广泛用于学习和生成离散分布，但其生成过程固有顺序性制约了效率。本文在连续时间马尔可夫链（CTMC）框架下，对主流的τ-跃迁算法进行并行化改进。通过利用τ-跃迁的连续时间随机积分形式与皮卡迭代方法，实现了时间维度上的并行采样加速，并证明了算法具有指数阶-阶乘收敛性。在吸收态设定下，τ-跃迁的整体时间复杂度从O(d log S)降低至O(log(d log S) · log d)，相对于NFE。实验表明，该方法在合成数据与真实数据上均表现稳定加速：合成分布最快提速7–9倍；图像与文本任务中保持相同生成质量，仅需50%的NFE，且单卡运行时间提速1.45–1.86倍。本研究拓展了离散扩散模型在高效并行推断中的潜力，对分子结构生成、语言生成等应用具有重要意义。
原文摘要 · Abstract (English)
Discrete diffusion models are widely used for learning and generating discrete distributions. As the generation process is inherently sequential, the acceleration of sampling is of significant importance. In this work, we parallelize the mainstream $τ$-leaping algorithm for absorbing discrete diffusion in a Continuous-Time Markov Chain (CTMC) framework. By leveraging the continuous-time stochastic integral form of the $τ$-leaping algorithm and the Picard iteration method, we achieve parallel-in-time sampling acceleration and provide a proof of exponential-factorial convergence for our algorithm. We improve the overall time complexity of $τ$-leaping under absorbing settings from ${\mathcal{O}}(d \log S)$ to ${\mathcal{O}}(\log (d\log S)\cdot \log d)$ with respect to NFE. Empirically, our method shows consistent acceleration across synthetic and real-data settings. The new sampler achieves at most $7$--$9\times$ runtime speedup for synthetic distribution, and maintains the same quality with $50\%$ fewer NFE and $1.45$--$1.86\times$ runtime speedups in image/text tasks on a single GPU. Our research expands the potential of discrete diffusion models for efficient parallel inference, with broader implications for applications such as molecular structure and language generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。