无需真实图像,1步生成高质量图像,解决文本对齐与多样性矛盾。
Few-Step Diffusion via Score identity Distillation

- 基于分数一致性蒸馏,无须真实图像即可训练单步生成器。
- 在1024x1024分辨率下对SDXL实现领先性能,支持1~3步生成。
- 提出零CFG与反向CFG策略,提升多样性且不损失文本对齐度。
扩散蒸馏已成为加速文生图模型的重要方法,通过将预训练得分网络蒸馏为一至几步生成器。现有方法在高分辨率模型如Stable Diffusion XL(SDXL)上表现良好,但通常依赖真实或教师合成图像,且使用无分类器引导(CFG)时存在文本-图像对齐与生成多样性之间的持续权衡。本文提出优化的分数一致性蒸馏(SiD)框架,实现无需数据的一步蒸馏,并拓展至少步生成。理论分析表明,匹配所有生成步骤输出的均匀混合分布可逼近数据分布,从而避免步骤专属网络,可无缝集成至现有流程。针对真实图文对可用情况,引入基于扩散GAN的对抗损失,并提出两种新引导策略:零CFG(教师禁用CFG,伪分数网络移除文本条件)与反向CFG(伪分数网络应用负向CFG),有效提升多样性且不牺牲对齐。在SD1.5和SDXL上的大量实验验证了其在单步与少步生成中的领先性能,且在无真实图像时仍具鲁棒性。我们将在GitHub开源高效PyTorch实现及蒸馏生成器。
原文摘要 · Abstract (English)
Diffusion distillation has emerged as a promising strategy for accelerating text-to-image (T2I) diffusion models by distilling a pretrained score network into a one- or few-step generator. While existing methods have made notable progress, they often rely on real or teacher-synthesized images to perform well when distilling high-resolution T2I diffusion models such as Stable Diffusion XL (SDXL), and their use of classifier-free guidance (CFG) introduces a persistent trade-off between text-image alignment and generation diversity. We address these challenges by optimizing Score identity Distillation (SiD) -- a data-free, one-step distillation framework -- for few-step generation. Backed by theoretical analysis that justifies matching a uniform mixture of outputs from all generation steps to the data distribution, our few-step distillation algorithm avoids step-specific networks and integrates seamlessly into existing pipelines, achieving state-of-the-art performance on SDXL at 1024x1024 resolution. To mitigate the alignment-diversity trade-off when real text-image pairs are available, we introduce a Diffusion GAN-based adversarial loss applied to the uniform mixture and propose two new guidance strategies: Zero-CFG, which disables CFG in the teacher and removes text conditioning in the fake score network, and Anti-CFG, which applies negative CFG in the fake score network. This flexible setup improves diversity without sacrificing alignment. Comprehensive experiments on SD1.5 and SDXL demonstrate state-of-the-art performance in both one-step and few-step generation settings, along with robustness to the absence of real images. Our efficient PyTorch implementation, along with the resulting one- and few-step distilled generators, will be released publicly as a separate branch at https://github.com/mingyuanzhou/SiD-LSG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。