通过稀疏训练提升扩散模型效率，兼顾性能与资源消耗。
Sparse-to-Sparse Training of Diffusion Models
- 从零开始训练稀疏扩散模型，采用静态、动态等三种稀疏策略。
- 稀疏模型参数和计算量大幅减少，性能却接近甚至超过密集模型。
- 为扩散模型稀疏训练提供可复现的高效配置方案，适合资源受限场景。
扩散模型（DMs）在图像生成等任务中表现卓越，但训练与推理成本高昂。现有研究多聚焦于推理加速，本文首次提出稀疏到稀疏训练范式，旨在同时优化训练与推理效率。我们在六个数据集上，基于潜空间扩散模型（Latent Diffusion）和ChiroDiff，采用三种方法（Static-DM、RigL-DM、MagRan-DM）从零训练稀疏扩散模型，探索稀疏性对性能的影响。实验表明，稀疏模型在显著降低可训练参数量和浮点运算量（FLOPs）的同时，仍能匹配甚至超越密集模型的生成质量，并识别出安全有效的稀疏训练配置范围。
原文摘要 · Abstract (English)
Diffusion models (DMs) are a powerful type of generative models that have achieved state-of-the-art results in various image synthesis tasks and have shown potential in other domains, such as natural language processing and temporal data modeling. Despite their stable training dynamics and ability to produce diverse high-quality samples, DMs are notorious for requiring significant computational resources, both in the training and inference stages. Previous work has focused mostly on increasing the efficiency of model inference. This paper introduces, for the first time, the paradigm of sparse-to-sparse training to DMs, with the aim of improving both training and inference efficiency. We focus on unconditional generation and train sparse DMs from scratch (Latent Diffusion and ChiroDiff) on six datasets using three different methods (Static-DM, RigL-DM, and MagRan-DM) to study the effect of sparsity in model performance. Our experiments show that sparse DMs are able to match and often outperform their Dense counterparts, while substantially reducing the number of trainable parameters and FLOPs. We also identify safe and effective values to perform sparse-to-sparse training of DMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。