统一缓存与剪枝，提升视频生成效率与画质。
UniCP: A Unified Caching and Pruning Framework for Efficient Video Generation
- 动态调整缓存窗口，适应不同时间步的误差变化。
- 通过剪枝减少冗余注意力，提升计算效率。
- 适合追求高效高保真视频生成的研究者与开发者。
扩散变换器（DiT）在视频生成中表现优异，但因注意力机制存在二次复杂度，面临显著计算挑战。值得注意的是，相邻扩散步骤间的注意力差异呈现U形分布。现有方法虽利用此特性进行注意力块缓存，但仍存在突发误差峰值和较大偏差问题。为此，我们提出UniCP——一种统一的缓存与剪枝框架，用于高效视频生成。UniCP通过误差感知动态缓存窗口（EDCW），根据不同时间步和注意力块动态调整缓存范围，以应对误差突变；结合基于PCA的切片（PCAS）与动态权重切换（DWS），实现冗余注意力组件的剪枝，并支持剪枝输出与缓存输出之间的动态切换。通过优化时空维度的计算开销，UniCP在保持视频细节保真度的同时显著提升效率。实验表明，UniCP在性能与效率上均优于现有方法。
原文摘要 · Abstract (English)
Diffusion Transformers (DiT) excel in video generation but encounter significant computational challenges due to the quadratic complexity of attention. Notably, attention differences between adjacent diffusion steps follow a U-shaped pattern. Current methods leverage this property by caching attention blocks, however, they still struggle with sudden error spikes and large discrepancies. To address these issues, we propose UniCP a unified caching and pruning framework for efficient video generation. UniCP optimizes both temporal and spatial dimensions through. Error Aware Dynamic Cache Window (EDCW): Dynamically adjusts cache window sizes for different blocks at various timesteps, adapting to abrupt error changes. PCA based Slicing (PCAS) and Dynamic Weight Shift (DWS): PCAS prunes redundant attention components, and DWS integrates caching and pruning by enabling dynamic switching between pruned and cached outputs. By adjusting cache windows and pruning redundant components, UniCP enhances computational efficiency and maintains video detail fidelity. Experimental results show that UniCP outperforms existing methods in both performance and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。