arXiv:2501.11211cs.ARcs.CV2025-01中稿 · publication at the…被引 28

利用时间相似性压缩扩散模型计算,提升生成速度与能效。

Ditto: Accelerating Diffusion Model via Temporal Value Similarity

  • 通过分析时间步间值的高相似性,仅用低比特差值处理后续步骤。
  • 相比其他加速器,硬件实现达1.5倍加速与17.74%能效提升。
  • 适合需要高效图像生成的实时应用或边缘设备部署。

扩散模型在图像生成任务中表现优异,但其迭代结构带来显著计算开销。我们分析该结构发现,扩散模型相邻时间步间值高度相似,连续步间差异较小。基于此,我们提出Ditto算法,利用时间相似性与量化技术提升效率:对初始时间步执行全精度运算,后续步骤仅处理差值,并结合层操作分布特性优化执行流程,降低内存开销。同时设计专用硬件加速器Ditto,充分挖掘算法动态特性。实验表明,该硬件相比其他加速器最高实现1.5倍速度提升,能耗降低17.74%。

原文摘要 · Abstract (English)

Diffusion models achieve superior performance in image generation tasks. However, it incurs significant computation overheads due to its iterative structure. To address these overheads, we analyze this iterative structure and observe that adjacent time steps in diffusion models exhibit high value similarity, leading to narrower differences between consecutive time steps. We adapt these characteristics to a quantized diffusion model and reveal that the majority of these differences can be represented with reduced bit-width, and even zero. Based on our observations, we propose the Ditto algorithm, a difference processing algorithm that leverages temporal similarity with quantization to enhance the efficiency of diffusion models. By exploiting the narrower differences and the distributive property of layer operations, it performs full bit-width operations for the initial time step and processes subsequent steps with temporal differences. In addition, Ditto execution flow optimization is designed to mitigate the memory overhead of temporal difference processing, further boosting the efficiency of the Ditto algorithm. We also design the Ditto hardware, a specialized hardware accelerator, fully exploiting the dynamic characteristics of the proposed algorithm. As a result, the Ditto hardware achieves up to 1.5x speedup and 17.74% energy saving compared to other accelerators.

扩散模型加速量化硬件

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。