提出新型缓存策略，让视频生成更高效清晰。
Past- and Future-Informed KV Cache Policy with Salience Estimation in Autoregressive Video Diffusion
- 根据过去和未来信息动态评估令牌重要性，智能筛选缓存
- 减少缓存容量与内存占用，推理速度提升显著
- 适合长时序视频生成，兼顾质量与效率
视频生成对数字媒体创作至关重要，自回归视频生成技术已显著提升实时合成效率。然而，现有方法普遍依赖启发式键值缓存策略，忽视了长期生成中令牌重要性的差异，导致关键时空信息丢失及冗余无效缓存累积，进而降低生成质量和效率。为此，我们观察到令牌对视频生成的贡献具有高度时间异质性，提出一种新的过去与未来信息感知的键值缓存策略（PaFu-KV）。该策略通过从双向教师模型中蒸馏出轻量级显著性估计头，评估令牌显著性分数，使缓存保留有信息量的令牌并丢弃不相关项。该策略在缩小缓存容量、降低推理时内存开销的同时，实现更优的质量-效率权衡。大量实验表明，本方法在保持高保真度视频生成质量的前提下，实现加速推理，推动更高效的长时程视频生成。代码将在论文接受后公开。
原文摘要 · Abstract (English)
Video generation is pivotal to digital media creation, and recent advances in autoregressive video generation have markedly enhanced the efficiency of real-time video synthesis. However, existing approaches generally rely on heuristic KV Cache policies, which ignore differences in token importance in long-term video generation. This leads to the loss of critical spatiotemporal information and the accumulation of redundant, invalid cache, thereby degrading video generation quality and efficiency. To address this limitation, we first observe that token contributions to video generation are highly time-heterogeneous and accordingly propose a novel Past- and Future-Informed KV Cache Policy (PaFu-KV). Specifically, PaFu-KV introduces a lightweight Salience Estimation Head distilled from a bidirectional teacher to estimate salience scores, allowing the KV cache to retain informative tokens while discarding less relevant ones. This policy yields a better quality-efficiency trade-off by shrinking KV cache capacity and reducing memory footprint at inference time. Extensive experiments on benchmarks demonstrate that our method preserves high-fidelity video generation quality while enables accelerated inference, thereby enabling more efficient long-horizon video generation. Our code will be released upon paper acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。