提出两种修正采样器,显著降低离散流模型的采样误差。
Corrected Samplers for Discrete Flow Models
- 基于离散流框架,分析采样器误差并设计时间与位置修正方法。
- 在文本到图像生成中实现更高质量结果,且推理时间更短。
- 无需额外计算成本,适合需要高效生成的场景。
离散流模型(DFM)被提出用于学习有限状态空间上的数据分布,为离散扩散模型提供了一种灵活的替代框架。近期研究提出了针对离散扩散模型的采样器,如tau-leaping和Euler求解器,但这些方法需大量迭代以控制离散化误差,因为转移速率在时间上冻结,并在每个时间区间内基于初始状态计算。此外,这些采样器的理论结果通常依赖于转移率的有界性或特定源分布。为此,本文在离散流模型框架下,建立了无需任何转移率或源分布限制的非渐近离散化误差界。通过分析Euler采样器的一步下界,我们提出了两种修正采样器:时间修正采样器和位置修正采样器,可显著降低tau-leaping和Euler求解器的离散化误差,且几乎不增加计算开销。我们严格证明了位置修正采样器的复杂度低于现有并行采样器。在模拟和文本到图像生成任务中验证了所提方法的有效性,实现了更高生成质量与更短推理时间。代码见https://github.com/WanZhengyan/Corrected-Samplers-for-Discrete-Flow-Models。
原文摘要 · Abstract (English)
Discrete flow models (DFMs) have been proposed to learn the data distribution on finite state space, offering a flexible framework as an alternative to discrete diffusion models. A line of recent work has studied samplers for discrete diffusion models, such as tau-leaping and Euler solver. However, these samplers require a large number of iterations to control discretization error, since the transition rates are frozen in time and evaluated at the initial state within each time interval. Moreover, theoretical results for these samplers often require boundedness conditions of the transition rate or they focus on a specific type of source distributions. To address those limitations, we establish non-asymptotic discretization error bounds for those samplers without any restriction on transition rates and source distributions, under the framework of discrete flow models. Furthermore, by analyzing a one-step lower bound of the Euler sampler, we propose two corrected samplers: \textit{time-corrected sampler} and \textit{location-corrected sampler}, which can reduce the discretization error of tau-leaping and Euler solver with almost no additional computational cost. We rigorously show that the location-corrected sampler has a lower complexity than existing parallel samplers. We validate the effectiveness of the proposed method by achieving better generation quality with reduced inference time on simulations and text-to-image generation tasks. Code can be found in https://github.com/WanZhengyan/Corrected-Samplers-for-Discrete-Flow-Models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。