arXiv:2503.05156cs.CV2025-03ICCV被引 17

通过优化缓存梯度,让扩散Transformer生成更准更快。

Accelerating Diffusion Transformer via Gradient-Optimized Cache

  • 动态计算缓存与重算特征的梯度差,逐步修正误差。
  • 50%缓存时,图像质量指标提升26.3%,速度不变。
  • 适合追求高效高质图像生成的研究者和开发者。

特征缓存已成为加速扩散变压器(DiT)采样的有效策略,通过复用时间特征实现。然而,该方法面临两大挑战:(1) 缓存块超过50%时,累积误差显著降低生成质量;(2) 现有误差补偿方法忽略缓存过程中的动态扰动模式,导致纠错效果不佳。为此,我们提出梯度优化缓存(GOC),包含两项创新:(1) 缓存梯度传播:通过梯度队列动态计算缓存与重算特征间的梯度差异,加权后传递至后续步骤,直接补偿缓存引入的近似误差;(2) 转折点感知优化:通过分析特征变化模式,识别去噪轨迹方向转变的关键转折点,使梯度更新与检测阶段对齐,避免纠错过程中出现冲突梯度方向。在ImageNet上的大量实验表明,GOC在效率与质量间表现优异。当缓存比例为50%时,相比基线DiT,IS提升26.3%(达216.28),FID降低43%(降至3.907),且计算成本保持一致。该优势在多种缓存比例下均持续存在,展现出对不同加速需求的鲁棒适应性。代码已公开于https://github.com/qiujx0520/GOC_ICCV2025.git。

原文摘要 · Abstract (English)

Feature caching has emerged as an effective strategy to accelerate diffusion transformer (DiT) sampling through temporal feature reuse. It is a challenging problem since (1) Progressive error accumulation from cached blocks significantly degrades generation quality, particularly when over 50\% of blocks are cached; (2) Current error compensation approaches neglect dynamic perturbation patterns during the caching process, leading to suboptimal error correction. To solve these problems, we propose the Gradient-Optimized Cache (GOC) with two key innovations: (1) Cached Gradient Propagation: A gradient queue dynamically computes the gradient differences between cached and recomputed features. These gradients are weighted and propagated to subsequent steps, directly compensating for the approximation errors introduced by caching. (2) Inflection-Aware Optimization: Through statistical analysis of feature variation patterns, we identify critical inflection points where the denoising trajectory changes direction. By aligning gradient updates with these detected phases, we prevent conflicting gradient directions during error correction. Extensive evaluations on ImageNet demonstrate GOC's superior trade-off between efficiency and quality. With 50\% cached blocks, GOC achieves IS 216.28 (26.3\% higher) and FID 3.907 (43\% lower) compared to baseline DiT, while maintaining identical computational costs. These improvements persist across various cache ratios, demonstrating robust adaptability to different acceleration requirements. Code is available at https://github.com/qiujx0520/GOC_ICCV2025.git.

扩散模型加速生成特征缓存图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。