arXiv:2510.08669cs.LGcs.AI2025-10被引 17

通过频率感知缓存加速扩散模型推理,显著降低计算成本。

FreqCa: Accelerating Diffusion Models via Frequency-Aware Caching

  • 按频域特性区分高低频特征,分别采用相似性复用与连续性预测
  • 在多个模型上实现99%内存节省,推理速度提升2.1倍以上
  • 适合需要高效图像生成与编辑的部署场景

扩散模型的推理成本过高,现有特征缓存方法依赖相邻时间步特征的连续性,但这一假设并不总成立。本文从频域分析发现:低频成分(决定图像结构)相似性高但连续性差,高频成分(决定细节)连续性强但相似性差。据此提出频率感知缓存(FreqCa),对低频特征基于相似性复用,对高频特征使用二阶赫米特插值器基于连续性预测。此外,提出缓存累积残差特征(CRF)替代全层特征,将缓存内存减少99%。在FLUX.1-dev、FLUX.1-Kontext-dev、Qwen-Image和Qwen-Image-Edit上的实验表明,该方法在生成与编辑任务中均有效。代码将在补充材料及GitHub公开。

原文摘要 · Abstract (English)

The application of diffusion transformers is suffering from their significant inference costs. Recently, feature caching has been proposed to solve this problem by reusing features from previous timesteps, thereby skipping computation in future timesteps. However, previous feature caching assumes that features in adjacent timesteps are similar or continuous, which does not always hold in all settings. To investigate this, this paper begins with an analysis from the frequency domain, which reveal that different frequency bands in the features of diffusion models exhibit different dynamics across timesteps. Concretely, low-frequency components, which decide the structure of images, exhibit higher similarity but poor continuity. In contrast, the high-frequency bands, which decode the details of images, show significant continuity but poor similarity. These interesting observations motivate us to propose Frequency-aware Caching (FreqCa) which directly reuses features of low-frequency components based on their similarity, while using a second-order Hermite interpolator to predict the volatile high-frequency ones based on its continuity. Besides, we further propose to cache Cumulative Residual Feature (CRF) instead of the features in all the layers, which reduces the memory footprint of feature caching by 99%. Extensive experiments on FLUX.1-dev, FLUX.1-Kontext-dev, Qwen-Image, and Qwen-Image-Edit demonstrate its effectiveness in both generation and editing. Codes are available in the supplementary materials and will be released on GitHub.

扩散模型特征缓存频率分析推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。