用预训练扩散模型的中间状态做特征,一步实现高效高质图像生成。
Teacher-Feature Drifting: One-Step Diffusion Distillation with Pretrained Diffusion Representations

- 直接利用教师模型中间层特征,省去额外特征提取网络。
- 在ImageNet和SDXL上分别达FID 1.58和18.4,生成质量优异。
- 方法简洁,适合追求高效部署的生成模型研究者。
从预训练扩散模型和流匹配模型采样通常需要大量前向传播才能生成多样且高质量的图像。现有蒸馏方法常依赖多个辅助网络、精心设计的训练阶段或复杂优化流程。本文重新审视最近提出的漂移模型目标,表明单个漂移损失可直接用于简化一步蒸馏。关键观察是:预训练扩散教师模型本身已提供强表征空间。与原始漂移模型依赖额外预训练特征提取器不同,我们使用教师模型的中间隐藏状态作为特征表示,无需训练或引入额外表征网络,同时保持语义有意义的特征几何结构。此外,我们引入轻量级模式覆盖损失,缓解蒸馏过程中的模式崩溃问题,促使学生生成器覆盖教师支持的多样化区域。在ImageNet和SDXL上的大量实验表明,该方法实现了高效的一步生成,图像质量和多样性具有竞争力,在ImageNet-64×64上达到FID 1.58,SDXL上为18.4,同时显著简化了整体蒸馏框架。
原文摘要 · Abstract (English)
Sampling from pretrained diffusion and flow-matching models typically requires many forward passes to generate diverse and high-fidelity images. Existing distillation methods often rely on multiple auxiliary networks, carefully designed training stages, or complex optimization pipelines. In this work, we revisit the recently proposed Drifting Model objective and show that a single drifting loss can be directly used to simplify one step distillation. A key observation is that the pretrained diffusion teacher itself already provides a strong representation space. Unlike the original Drifting Model, which relies on an additional pretrained feature extractor, we use intermediate hidden states of the pretrained teacher model as the feature representation. This removes the need for training or introducing an extra representation network while preserving a semantically meaningful feature geometry for drifting. Furthermore, we introduce a lightweight mode coverage loss to mitigate mode collapse during distillation and encourage the student generator to cover diverse teacher-supported regions. Extensive experiments on ImageNet and SDXL demonstrate that our method achieves efficient one step generation with competitive image quality and diversity, achieving FID scores of 1.58 on ImageNet-64$\times$64 and 18.4 on SDXL, while substantially simplifying the overall distillation framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。