无需训练即可生成超长视频,解决重复与运动停滞问题。
Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compression
- 用深层缓存与参与式压缩管理注意力历史,避免信息冗余
- 5秒训练可生成60秒以上视频,动态性提升显著
- 适合需要实时长视频生成的场景,如直播、虚拟角色
自回归视频扩散模型已实现帧流实时生成,但现有方法仍存在时间重复、运动漂移和减速问题。我们发现直接套用StreamingLLM式的注意力缓存机制会导致图像质量下降和运动停滞。为此提出Deep Forcing,包含两个无需训练的机制:1)深度缓存将滑动窗口一半用于持久化缓存标记,并重新对齐其时间位置编码相位,稳定长期生成中的全局上下文;2)参与式压缩基于重要性感知的键值缓存剪枝,仅保留近期注意力活跃的标记,安全丢弃冗余且退化的历史信息,从而在长序列生成中最小化误差累积。两者结合使模型实现超过12倍外推(如5秒训练扩展至60秒以上生成),在图像质量上优于LongLive,美学质量优于RollingForcing,几乎保持整体一致性,动态性显著提升,同时维持实时生成。结果表明,无需训练的键值缓存管理可达到甚至超越基于训练的方法在自回归流式长视频生成中的表现。
原文摘要 · Abstract (English)
Recent advances in autoregressive video diffusion have enabled real-time frame streaming, yet existing solutions still suffer from temporal repetition, drift, and motion deceleration. We find that naively applying StreamingLLM-style attention sinks to video diffusion leads to fidelity degradation and motion stagnation. To overcome this, we introduce Deep Forcing, which consists of two training-free mechanisms that address this without any fine-tuning. Specifically, 1) Deep Sink dedicates half of the sliding window to persistent sink tokens and re-aligns their temporal RoPE phase to the current timeline, stabilizing global context during long rollouts. 2) Participative Compression performs importance-aware KV cache pruning that preserves only tokens actively participating in recent attention while safely discarding redundant and degraded history, minimizing error accumulation under out-of-distribution length generation. Together, these components enable over 12x extrapolation (e.g. 5s-trained to 60s+ generation) with better imaging quality than LongLive, better aesthetic quality than RollingForcing, almost maintaining overall consistency, and substantial gains in dynamic degree, all while maintaining real-time generation. Our results demonstrate that training-free KV-cache management can match or exceed training-based approaches for autoregressively streaming long-video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。