提出无偏梯度估计方法，提升单步扩散蒸馏生成质量
VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation
- 构建变分扩散上界，实现无偏梯度估计
- 相比Diff-Instruct，生成质量更高，训练更稳定高效
- 适合追求高质量单步生成的扩散模型研究者
近期的扩散蒸馏方法将千步教师扩散模型压缩为单步学生生成器，同时保持样本质量。现有方法大多通过学生模型的得分函数近似扩散散度梯度，该得分函数通过去噪得分匹配（DSM）学习。由于DSM训练不完美，梯度估计存在偏差，导致性能不佳。本文提出VarDiU（发音/va:rdju:/），一种变分扩散上界，可直接用于扩散蒸馏并提供无偏梯度估计。实验表明，使用该目标的方法在生成质量上优于Diff-Instruct，且训练过程更高效、更稳定。
原文摘要 · Abstract (English)
Recently, diffusion distillation methods have compressed thousand-step teacher diffusion models into one-step student generators while preserving sample quality. Most existing approaches train the student model using a diffusive divergence whose gradient is approximated via the student's score function, learned through denoising score matching (DSM). Since DSM training is imperfect, the resulting gradient estimate is inevitably biased, leading to sub-optimal performance. In this paper, we propose VarDiU (pronounced /va:rdju:/), a Variational Diffusive Upper Bound that admits an unbiased gradient estimator and can be directly applied to diffusion distillation. Using this objective, we compare our method with Diff-Instruct and demonstrate that it achieves higher generation quality and enables a more efficient and stable training procedure for one-step diffusion distillation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。