用离散化与哈密顿转移提升扩散模型效率,逼近最优采样速度。
Almost Linear Convergence under Minimal Score Assumptions: Quantized Transition Diffusion
- 将连续数据离散化为编码空间,通过汉明距离实现长程跳跃转移。
- 理论证明在最小得分假设下,期望仅需 O(d ln²(d/ε)) 次得分评估即可逼近目标分布。
- 适合关注生成模型高效推理与理论突破的研究者,尤其对扩散模型优化者有参考价值。
连续扩散模型在多个领域展现出卓越的数据生成能力,但其效率受两大限制:(1)前向马尔可夫过程的局部邻接结构,限制了数据空间中的长程转移;(2)时变反向去噪过程模拟带来的固有偏差。为此,我们提出量化转移扩散(QTD),结合数据量化与离散扩散动态。方法首先通过直方图近似和二进制编码将连续数据分布 $p_*$ 转换为离散分布 $q_*$,在结构化的离散潜在空间中实现高效表示。设计基于汉明距离的连续时间马尔可夫链(CTMC)作为前向过程,天然支持原数据空间中的长程移动。针对反向采样,引入“截断均匀化”技术模拟反向CTMC,可在最小得分假设下无偏生成 $q_*$。通过新颖的逆向CTMC KL动态分析,我们证明QTD期望仅需 $O(d\ln^2(d/ε))$ 次得分评估,即可在 $ε$ 误差内逼近 $d$ 维目标分布 $p_*$。该方法不仅达到当前最优推理效率,更通过统一离散与连续扩散范式,推动了生成建模的理论基础发展。
原文摘要 · Abstract (English)
Continuous diffusion models have demonstrated remarkable performance in data generation across various domains, yet their efficiency remains constrained by two critical limitations: (1) the local adjacency structure of the forward Markov process, which restricts long-range transitions in the data space, and (2) inherent biases introduced during the simulation of time-inhomogeneous reverse denoising processes. To address these challenges, we propose Quantized Transition Diffusion (QTD), a novel approach that integrates data quantization with discrete diffusion dynamics. Our method first transforms the continuous data distribution $p_*$ into a discrete one $q_*$ via histogram approximation and binary encoding, enabling efficient representation in a structured discrete latent space. We then design a continuous-time Markov chain (CTMC) with Hamming distance-based transitions as the forward process, which inherently supports long-range movements in the original data space. For reverse-time sampling, we introduce a \textit{truncated uniformization} technique to simulate the reverse CTMC, which can provably provide unbiased generation from $q_*$ under minimal score assumptions. Through a novel KL dynamic analysis of the reverse CTMC, we prove that QTD can generate samples with $O(d\ln^2(d/ε))$ score evaluations in expectation to approximate the $d$--dimensional target distribution $p_*$ within an $ε$ error tolerance. Our method not only establishes state-of-the-art inference efficiency but also advances the theoretical foundations of diffusion-based generative modeling by unifying discrete and continuous diffusion paradigms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。