针对扩散模型推理速度慢问题，提出按频率动态缓存方案，提速近2.5倍且画质接近原版。
Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers
- 根据时间、深度、特征三维度差异，动态调整缓存策略
- 在512x512分辨率下实现2.46倍加速，画质损失极小
- 无需训练、即插即用，适合所有扩散变压器架构
扩散变换器（DiTs）已成为高质量图像与视频生成的主流架构，但其迭代去噪过程在推理阶段带来巨大计算开销。现有缓存方法通过复用不同时间步的中间结果来加速，却普遍将去噪过程视为均匀分布，忽略其在时间、深度和特征维度上的非均匀性。本文发现三个非均匀性维度：（1）时间——不同去噪阶段对缓存误差敏感度差异显著；（2）深度——连续缓存决策导致误差累积；（3）特征——隐藏状态各分量具有异构的时间动态特性。基于此，提出统一缓存框架SpectralCache，包含时间感知动态调度（TADS）、累计误差预算（CEB）与频域分解缓存（FDC）。在FLUX.1-schnell模型、512x512分辨率下，SpectralCache实现2.46倍加速，LPIPS为0.217，SSIM为0.727，相比TeaCache（2.12倍加速，LPIPS 0.215，SSIM 0.734）提升16%速度，且画质差距小于1%。该方法无需训练，可即插即用，兼容现有DiT架构。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) have emerged as the dominant architecture for high-quality image and video generation, yet their iterative denoising process incurs substantial computational cost during inference. Existing caching methods accelerate DiTs by reusing intermediate computations across timesteps, but they share a common limitation: treating the denoising process as uniform across time,depth, and feature dimensions. In this work, we identify three orthogonal axes of non-uniformity in DiT denoising: (1) temporal -- sensitivity to caching errors varies dramatically across the denoising trajectory; (2) depth -- consecutive caching decisions lead to cascading approximation errors; and (3) feature -- different components of the hidden state exhibit heterogeneous temporal dynamics. Based on these observations, we propose SpectralCache, a unified caching framework comprising Timestep-Aware Dynamic Scheduling (TADS), Cumulative Error Budgets (CEB), and Frequency-Decomposed Caching (FDC). On FLUX.1-schnell at 512x512 resolution, SpectralCache achieves 2.46x speedup with LPIPS 0.217 and SSIM 0.727, outperforming TeaCache (2.12x, LPIPS 0.215, SSIM 0.734) by 16% in speed while maintaining comparable quality (LPIPS difference < 1%). Our approach is training-free, plug-and-play, and compatible with existing DiT architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。