QuantCache通过分层缓存与自适应量化，实现视频生成模型的高效推理。
QuantCache: Adaptive Importance-Guided Quantization with Hierarchical Latent and Layer Caching for Video Generation
- 融合分层潜在缓存与自适应重要性量化，协同优化推理效率。
- 在Open-Sora上实现6.72倍延迟加速，生成质量损失极小。
- 适合部署于资源受限设备的视频生成模型推理加速。
最近，扩散变压器（DiTs）已成为视频生成的主导架构，在性能上超越基于U-Net的模型。然而，DiTs的增强能力带来了显著的计算和内存开销，阻碍了其在资源受限设备上的部署。现有的加速技术如量化和缓存机制虽有一定效果，但通常单独使用，难以充分应对DiT架构的复杂性。本文提出QuantCache，一种无需训练的推理加速框架，联合优化分层潜在缓存、自适应重要性引导量化和结构冗余感知剪枝。QuantCache在Open-Sora上实现了6.72倍的端到端延迟加速，生成质量损失微小。在多个视频生成基准测试中均验证了方法的有效性，为高效DiT推理树立了新标准。代码与模型将公开于https://github.com/JunyiWuCode/QuantCache。
原文摘要 · Abstract (English)
Recently, Diffusion Transformers (DiTs) have emerged as a dominant architecture in video generation, surpassing U-Net-based models in terms of performance. However, the enhanced capabilities of DiTs come with significant drawbacks, including increased computational and memory costs, which hinder their deployment on resource-constrained devices. Current acceleration techniques, such as quantization and cache mechanism, offer limited speedup and are often applied in isolation, failing to fully address the complexities of DiT architectures. In this paper, we propose QuantCache, a novel training-free inference acceleration framework that jointly optimizes hierarchical latent caching, adaptive importance-guided quantization, and structural redundancy-aware pruning. QuantCache achieves an end-to-end latency speedup of 6.72$\times$ on Open-Sora with minimal loss in generation quality. Extensive experiments across multiple video generation benchmarks demonstrate the effectiveness of our method, setting a new standard for efficient DiT inference. The code and models will be available at https://github.com/JunyiWuCode/QuantCache.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。