通过智能复用冗余计算块，实现无需训练的扩散模型加速
CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformers
- 按贡献度选择性缓存与复用变压器块输出
- 平均提速2.0倍，生成质量基本不变
- 适合追求推理效率的视觉生成应用
扩散变换器（DiT）在图像生成中表现卓越，但其迭代去噪过程与大容量导致高推理成本。现有研究发现，DiT的迭代过程存在大量跨步骤冗余计算。为此，本文提出CorGi（贡献引导的块级间隔缓存）——一种无需训练的DiT推理加速框架，通过选择性复用不同去噪步骤间变压器块的输出来减少冗余计算，同时保持生成质量。针对文生图任务，进一步提出CorGi+，利用每个块的交叉注意力图识别显著标记，仅对关键区域进行部分注意力更新以保护重要物体细节。在主流DiT模型上的评估表明，CorGi与CorGi+平均实现2.0倍加速，同时保持高质量生成结果。
原文摘要 · Abstract (English)
Diffusion transformer (DiT) achieves remarkable performance in visual generation, but its iterative denoising process combined with larger capacity leads to a high inference cost. Recent works have demonstrated that the iterative denoising process of DiT models involves substantial redundant computation across steps. To effectively reduce the redundant computation in DiT, we propose CorGi (Contribution-Guided Block-Wise Interval Caching), training-free DiT inference acceleration framework that selectively reuses the outputs of transformer blocks in DiT across denoising steps. CorGi caches low-contribution blocks and reuses them in later steps within each interval to reduce redundant computation while preserving generation quality. For text-to-image tasks, we further propose CorGi+, which leverages per-block cross-attention maps to identify salient tokens and applies partial attention updates to protect important object details. Evaluation on the state-of-the-art DiT models demonstrates that CorGi and CorGi+ achieve up to 2.0x speedup on average, while preserving high generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。