arXiv:2508.16212cs.CVcs.AI2025-08ICCV被引 17

不训练即可加速扩散模型,通过全局轨迹优化缓存复用。

OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models

  • 从采样轨迹视角设计缓存策略,全局分布复用计算
  • 动态去噪并过滤干扰,保持生成质量不变
  • 无需训练,适合快速部署扩散模型

扩散模型在图像合成与视频生成等生成任务中表现卓越,基于Transformer架构进一步提升了性能。然而,扩散Transformer因采样步数多、每步计算复杂,导致计算开销巨大,难以实现实时部署。本文提出OmniCache,一种无需训练的加速方法,利用去噪过程中的全局冗余特性。不同于依赖步骤间相似性、侧重复用后期采样的现有方法,我们的方法基于DIT模型的采样视角,系统分析采样轨迹,并将缓存复用策略战略性地分布于整个采样过程。这一全局视角使缓存计算在整个扩散轨迹中更高效利用,而非局限于有限片段。此外,在缓存复用过程中,动态估计对应噪声并加以过滤,降低其对采样方向的影响。大量实验表明,该方法在保持优异生成质量的同时显著加速采样过程,为基于扩散的生成模型提供了高效实用的部署方案。

原文摘要 · Abstract (English)

Diffusion models have emerged as a powerful paradigm for generative tasks such as image synthesis and video generation, with Transformer architectures further enhancing performance. However, the high computational cost of diffusion Transformers-stemming from a large number of sampling steps and complex per-step computations-presents significant challenges for real-time deployment. In this paper, we introduce OmniCache, a training-free acceleration method that exploits the global redundancy inherent in the denoising process. Unlike existing methods that determine caching strategies based on inter-step similarities and tend to prioritize reusing later sampling steps, our approach originates from the sampling perspective of DIT models. We systematically analyze the model's sampling trajectories and strategically distribute cache reuse across the entire sampling process. This global perspective enables more effective utilization of cached computations throughout the diffusion trajectory, rather than concentrating reuse within limited segments of the sampling procedure. In addition, during cache reuse, we dynamically estimate the corresponding noise and filter it out to reduce its impact on the sampling direction. Extensive experiments demonstrate that our approach accelerates the sampling process while maintaining competitive generative quality, offering a promising and practical solution for efficient deployment of diffusion-based generative models.

扩散模型缓存优化生成模型加速推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。