通过跳过重复的视频隐空间块，提升生成速度而不损失质量
Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recovery
- 识别并跳过视频帧间重复的隐空间块，减少计算量
- 引入注意力恢复机制，消除剪枝带来的视觉伪影，保持生成质量
- 无需额外训练，可直接集成，实测提速1.53倍至19.3FPS
当前视频生成模型因计算延迟高，难以用于实时应用。本文提出潜空间帧间剪枝与注意力恢复（LIPAR）框架，利用视频隐空间块的时间冗余性，检测并跳过重复块的重新计算。同时引入新型注意力恢复机制，近似被剪枝标记的注意力值，有效消除朴素剪枝导致的视觉伪影。实验表明，该方法使视频编辑吞吐量提升1.53倍，在NVIDIA RTX 4090上以1.3B Self-Forcing模型（4步去噪，FP16）实现平均19.3 FPS，且不降低生成质量，可无缝集成至模型而无需额外训练。本方法有效弥合了传统压缩算法与现代生成流水线之间的差距。
原文摘要 · Abstract (English)
Current video generation models suffer from high computational latency, making real-time applications prohibitively costly. In this paper, we address this limitation by exploiting the temporal redundancy inherent in video latent patches. To this end, we propose the Latent Inter-frame Pruning with Attention Recovery (LIPAR) framework, which detects and skips recomputing duplicated latent patches. Additionally, we introduce a novel Attention Recovery mechanism that approximates the attention values of pruned tokens, thereby removing visual artifacts arising from naively applying the pruning method. Empirically, our method increases video editing throughput by $1.53\times$, achieving an average of 19.3 FPS on an NVIDIA RTX 4090 with the 1.3B Self-Forcing model (4-step denoising, FP16). The proposed method does not compromise generation quality and can be seamlessly integrated with the model without additional training. Our approach effectively bridges the gap between traditional compression algorithms and modern generative pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。