通过缓存注意力机制，8步即可生成高质量视频，速度提升1.65至3.35倍。
CHAI: CacHe Attention Inference for text2video
- 引入跨推理缓存注意力，智能复用相似提示下的潜在特征。
- 仅需8个去噪步骤即可保持视频质量，缓存命中率高。
- 适合追求高速生成且不牺牲画质的视频生成研究者。
文本到视频扩散模型虽效果出色，但因3D潜在变量的串行去噪而速度缓慢。现有加速方法或需昂贵重训练，或依赖启发式步跳，随去噪步数减少，视频质量显著下降。本文提出CHAI，利用跨推理缓存来降低延迟并保持画质。我们引入缓存注意力（Cache Attention），有效关注跨推理潜在变量间共享的物体/场景。该选择性注意力机制可实现语义相关提示间的缓存潜变量高效复用，获得高缓存命中率。实验表明，使用缓存注意力仅需8个去噪步骤即可生成高质量视频。集成至系统后，CHAI相较基线OpenSora 1.2提速1.65倍至3.35倍，同时维持视频质量。
原文摘要 · Abstract (English)
Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。