压缩大模型，让扩散模型更快更省地生成图像
A Survey on Pre-Trained Diffusion Model Distillations
- 从输出、轨迹、对抗三方面系统梳理蒸馏方法
- 可实现少步生成，降低资源消耗，适合边缘部署
- 适合想快速了解扩散模型压缩技术的研究者
扩散模型（DMs）已成为生成式人工智能的主流方法，在文生图等任务中表现优异。然而，如 Stable Diffusion 这类实际应用的模型通常在海量数据上训练，需大量存储空间；生成高质量图像时需多次递归调用神经网络，导致采样过程计算成本高昂。为此，基于预训练扩散模型的蒸馏方法被广泛采用，以构建更小、更高效的模型，实现在低资源环境下的快速、少步生成。由于不同视角下发展出多种蒸馏方法，亟需从方法论角度进行系统性综述。本文从输出损失蒸馏、轨迹蒸馏和对抗蒸馏三个维度回顾现有方法，并讨论当前挑战与未来研究方向。
原文摘要 · Abstract (English)
Diffusion Models~(DMs) have emerged as the dominant approach in Generative Artificial Intelligence (GenAI), owing to their remarkable performance in tasks such as text-to-image synthesis. However, practical DMs, such as stable diffusion, are typically trained on massive datasets and thus usually require large storage. At the same time, many steps may be required, i.e., recursively evaluating the trained neural network, to generate a high-quality image, which results in significant computational costs during sample generation. As a result, distillation methods on pre-trained DM have become widely adopted practices to develop smaller, more efficient models capable of rapid, few-step generation in low-resource environment. When these distillation methods are developed from different perspectives, there is an urgent need for a systematic survey, particularly from a methodological perspective. In this survey, we review distillation methods through three aspects: output loss distillation, trajectory distillation and adversarial distillation. We also discuss current challenges and outline future research directions in the conclusion.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。