提出精确缓存机制,实现视频生成加速且不降质。
PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generation
- 通过低频差异检测步骤级冗余,精准跳过可复用计算
- 在Wan2.1-14B上实现平均2.6倍加速,质量无明显下降
- 适配多种模型,即插即用,适合高效视频生成场景
高计算成本和慢推理限制了视频生成模型的实际应用。尽管先前工作通过特征缓存加速生成过程,但常伴随显著质量下降。本文揭示问题源于无法区分真正冗余的特征,导致重要计算被错误跳过。为此,我们提出 extbf{PreciseCache},一种即插即用的框架,能精确检测并跳过真正冗余的计算,从而在不损失质量的前提下加速推理。具体包含两个组件:LFCache实现逐步缓存,通过计算当前步预测特征与前一缓存步的低频差异(LFD),实证表明LFD是有效的步骤级冗余度量,可准确识别可跳过的高度冗余步骤;BlockCache则在每个非跳过步骤内,以块为单位精确检测并跳过冗余计算。在多种骨干网络上的实验表明,PreciseCache表现优异,例如在Wan2.1-14B上实现平均2.6倍加速,且无明显质量损失。
原文摘要 · Abstract (English)
High computational costs and slow inference hinder the practical application of video generation models. While prior works accelerate the generation process through feature caching, they often suffer from notable quality degradation. In this work, we reveal that this issue arises from their inability to distinguish truly redundant features, which leads to the unintended skipping of computations on important features. To address this, we propose \textbf{PreciseCache}, a plug-and-play framework that precisely detects and skips truly redundant computations, thereby accelerating inference without sacrificing quality. Specifically, PreciseCache contains two components: LFCache for step-wise caching and BlockCache for block-wise caching. For LFCache, we compute the Low-Frequency Difference (LFD) between the prediction features of the current step and those from the previous cached step. Empirically, we observe that LFD serves as an effective measure of step-wise redundancy, accurately detecting highly redundant steps whose computation can be skipped through reusing cached features. To further accelerate generation within each non-skipped step, we propose BlockCache, which precisely detects and skips redundant computations at the block level within the network. Extensive experiments on various backbones demonstrate the effectiveness of our PreciseCache, such as achieving an average of $2.6\times$ speedup on Wan2.1-14B without noticeable quality loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。