arXiv:2511.08930cs.CVcs.AI2025-11被引 1

分层蒸馏让单步生成图像更清晰,兼顾结构与细节。

From Structure to Detail: Hierarchical Distillation for Efficient Diffusion Model

  • 先用轨迹蒸馏建结构骨架,再用分布蒸馏精修细节。
  • 单步生成图像FID达2.26,接近250步教师模型性能。
  • 适合追求高速高质图像生成的研究者和开发者。

扩散模型的推理延迟仍是其实时应用的关键瓶颈。现有基于轨迹和分布的步骤蒸馏方法存在根本性权衡:前者保留全局结构但损失高频细节,后者虽能提升保真度却易出现模式崩溃且训练不稳定。本文将二者整合为协同工作的分层蒸馏(HD)框架,不再将轨迹蒸馏作为最终生成器,而是用于构建结构“草图”,为后续分布蒸馏阶段提供近优初始化。该策略显著提升整体性能上限。为进一步优化质量,我们引入并改进对抗训练过程,提出自适应加权判别器(AWD),通过动态分配令牌权重聚焦局部瑕疵,实现高效细节修复。实验表明,该方法在多种任务上均达领先水平:在ImageNet $256 imes256$ 上,单步模型FID仅为2.26,媲美250步教师模型;在高分辨率文本到图像MJHQ基准上也表现优异,验证了其通用性。本方法建立了一种新的高保真、单步扩散模型范式。

原文摘要 · Abstract (English)

The inference latency of diffusion models remains a critical barrier to their real-time application. While trajectory-based and distribution-based step distillation methods offer solutions, they present a fundamental trade-off. Trajectory-based methods preserve global structure but act as a "lossy compressor", sacrificing high-frequency details. Conversely, distribution-based methods can achieve higher fidelity but often suffer from mode collapse and unstable training. This paper recasts them from independent paradigms into synergistic components within our novel Hierarchical Distillation (HD) framework. We leverage trajectory distillation not as a final generator, but to establish a structural ``sketch", providing a near-optimal initialization for the subsequent distribution-based refinement stage. This strategy yields an ideal initial distribution that enhances the ceiling of overall performance. To further improve quality, we introduce and refine the adversarial training process. We find standard discriminator structures are ineffective at refining an already high-quality generator. To overcome this, we introduce the Adaptive Weighted Discriminator (AWD), tailored for the HD pipeline. By dynamically allocating token weights, AWD focuses on local imperfections, enabling efficient detail refinement. Our approach demonstrates state-of-the-art performance across diverse tasks. On ImageNet $256\times256$, our single-step model achieves an FID of 2.26, rivaling its 250-step teacher. It also achieves promising results on the high-resolution text-to-image MJHQ benchmark, proving its generalizability. Our method establishes a robust new paradigm for high-fidelity, single-step diffusion models.

扩散模型图像生成蒸馏高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。