arXiv:2602.06849cs.LG2026-02被引 2

提出两种新型采样调度,提升离散扩散模型生成质量与效率

Improved Sampling Schedules for Discrete Diffusion Models

  • 基于热力学熵产生率设计恒定信息增益的采样路径
  • 在多领域实验中显著优于现有方法,且计算开销更低
  • 适合追求高效高质量序列生成的研究者与工程师

离散扩散模型在序列数据生成中表现出强大能力,但其反向过程的信息理论机制远不如连续版本清晰。本文从热力学熵产生角度分析反向过程动态,提出熵产生率作为信息生成的严格度量,并导出中间状态与数据分布间Wasserstein距离的上界。基于此,我们提出两种新型采样调度:熵离散调度(EDS)通过保持恒定的信息增益率实现均匀采样;Wasserstein离散调度(WDS)则以相等的Wasserstein距离步长进行采样。实验证明,这两种调度在合成数据、乐谱、视觉和语言建模等多个领域均显著优于当前最优策略,且在更低的计算成本下达成更优性能。

原文摘要 · Abstract (English)

Discrete diffusion models have emerged as a powerful paradigm for generative modeling on sequence data; however, the information-theoretic principles governing their reverse processes remain significantly less understood than those of their continuous counterparts. In this work, we bridge this gap by analyzing the reverse process dynamics through the lens of thermodynamic entropy production. We propose the entropy production rate as a rigorous proxy for quantifying information generation, deriving as a byproduct a bound on the Wasserstein distance between intermediate states and the data distribution. Leveraging these insights, we introduce two novel sampling schedules that are uniformly spaced with respect to their corresponding physics-inspired metrics: the Entropic Discrete Schedule (EDS), which is defined by maintaining a constant rate of information gain, and the Wasserstein Discrete Schedule (WDS), which is defined by taking equal steps in terms of the Wasserstein distance. We empirically demonstrate that our proposed schedules significantly outperform state-of-the-art strategies across diverse application domains, including synthetic data, music notation, vision and language modeling, consistently achieving superior performance at a lower computational budget.

扩散模型采样调度序列生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。