提出混合缓存框架,让视频生成更快更准
Adaptive Hybrid Caching for Efficient Text-to-Video Diffusion Model Acceleration
- 根据上下文动态选择缓存粒度,灵活平衡速度与质量
- 在万相14B和云骁视频上分别提速1.94倍和1.97倍
- 无需训练,适配多种视频扩散模型,提升推理效率
高效视频生成模型在多媒体内容生成中日益重要。基于Transformer架构与扩散过程的视频DiT模型已成为高质量视频生成的主流方法。然而,其多步迭代去噪过程带来高计算成本与推理延迟。缓存是广泛采用的优化方法,利用扩散过程中的冗余性,在不同粒度(如步骤、CFG、模块)上跳过计算。但现有缓存方法局限于单一粒度策略,难以灵活平衡生成质量与推理速度。本文提出MixCache,一种无需训练的缓存框架,用于高效视频DiT推理。它首先区分不同缓存策略间的干扰与边界,引入上下文感知的缓存触发机制以决定何时启用缓存,并设计自适应混合缓存决策策略,动态选择最优缓存粒度。大量实验表明,MixCache在多种模型上显著加速视频生成(如万相14B上达1.94×,云骁视频上达1.97×),同时优于基线方法,在生成质量与推理效率上均表现更优。
原文摘要 · Abstract (English)
Efficient video generation models are increasingly vital for multimedia synthetic content generation. Leveraging the Transformer architecture and the diffusion process, video DiT models have emerged as a dominant approach for high-quality video generation. However, their multi-step iterative denoising process incurs high computational cost and inference latency. Caching, a widely adopted optimization method in DiT models, leverages the redundancy in the diffusion process to skip computations in different granularities (e.g., step, cfg, block). Nevertheless, existing caching methods are limited to single-granularity strategies, struggling to balance generation quality and inference speed in a flexible manner. In this work, we propose MixCache, a training-free caching-based framework for efficient video DiT inference. It first distinguishes the interference and boundary between different caching strategies, and then introduces a context-aware cache triggering strategy to determine when caching should be enabled, along with an adaptive hybrid cache decision strategy for dynamically selecting the optimal caching granularity. Extensive experiments on diverse models demonstrate that, MixCache can significantly accelerate video generation (e.g., 1.94$\times$ speedup on Wan 14B, 1.97$\times$ speedup on HunyuanVideo) while delivering both superior generation quality and inference efficiency compared to baseline methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。