通过聚类感知的令牌剪枝，显著降低文生图扩散模型计算开销。
CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Models
- 基于噪声幅度识别每轮中关键变化的令牌
- 结合空间聚类与分布均衡，提升剪枝精度
- 在保留生成质量前提下降低50%-60%计算量
扩散模型已彻底革新生成任务，尤其在文本到图像合成领域；然而其迭代去噪过程需大量计算资源。本文提出一种新型加速策略，将令牌级剪枝与缓存技术相结合，以应对这一挑战。通过噪声相对幅度识别去噪过程中显著变化的令牌，并引入空间聚类与分布平衡机制优化令牌选择。实验表明，该方法在保持模型性能的前提下，实现50%-60%的计算成本降低，显著提升扩散模型效率。代码已开源：https://github.com/ada-cheng/CAT-Pruning。
原文摘要 · Abstract (English)
Diffusion models have revolutionized generative tasks, especially in the domain of text-to-image synthesis; however, their iterative denoising process demands substantial computational resources. In this paper, we present a novel acceleration strategy that integrates token-level pruning with caching techniques to tackle this computational challenge. By employing noise relative magnitude, we identify significant token changes across denoising iterations. Additionally, we enhance token selection by incorporating spatial clustering and ensuring distributional balance. Our experiments demonstrate reveal a 50%-60% reduction in computational costs while preserving the performance of the model, thereby markedly increasing the efficiency of diffusion models. The code is available at https://github.com/ada-cheng/CAT-Pruning
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。