提出直接匹配条件分布的蒸馏方法，加速离散扩散模型采样。
Distillation of Discrete Diffusion by Exact Conditional Distribution Matching
- 基于条件分布匹配设计蒸馏目标，无需近似模拟器。
- 实现单步与少步采样器的高效训练，降低采样计算量。
- 适合需要快速生成离散数据的研究者使用。
离散扩散模型是处理类别数据的强大生成模型，但通常需要大量函数求值才能生成一个样本，导致推理成本高昂。现有加速方法要么依赖近似模拟器（如τ-跳跃法），要么采用训练新学生模型和辅助网络的蒸馏方案，且目标函数为代理目标。本文提出一种简单而原理清晰的蒸馏替代方法，基于条件分布匹配。关键观察是：给定噪声状态下的干净数据反向条件分布 $p_{0 }(x_0 x_t)$ 可通过中间时间点进行马尔可夫分解，并能从边际密度比和已知前向连续时间马尔可夫链核中恢复。我们利用这一结构，定义了直接匹配预训练教师与低函数求值数（low-NFE）学生模型之间条件分布的蒸馏目标，适用于单步和少步采样器。
原文摘要 · Abstract (English)
Discrete diffusion models (DDMs) are a powerful class of generative models for categorical data, but they typically require many function evaluations for a single sample, making inference expensive. Existing acceleration methods either rely on approximate simulators, such as $τ$-leaping, or on distillation schemes that train new student models and auxiliary networks with proxy objectives. We propose a simple and principled distillation alternative based on \emph{conditional distribution matching}. Our key observation is that the reverse conditional distribution of clean data given a noisy state, $p_{0\mid t}(x_0 \mid x_t)$, admits a Markov decomposition through intermediate times and can be recovered from marginal density ratios and the known forward CTMC kernel. We exploit this structure to define distillation objectives that directly match conditional distributions between a pre-trained teacher and a low-NFE student, both for one-step and few-step samplers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。