提升长视频生成速度与连贯性,实现实时无界生成。
Long-Horizon Streaming Video Generation via Hybrid Attention with Decoupled Distillation

- 用轻量线性时序注意力保留滑动窗口外的长程依赖。
- 通过块稀疏注意力降低局部计算冗余,提升效率。
- 分阶段蒸馏策略确保模型稳定,适合实时视频生成场景。
流式视频生成(SVG)将预训练的双向视频扩散模型蒸馏为配备滑动窗口注意力(SWA)的自回归模型。然而,SWA在长视频生成中不可避免地丢失远距离历史,且计算开销仍是实时部署的关键挑战。本文提出 Hybrid Forcing,通过混合注意力设计联合优化时序信息保留与计算效率。首先引入轻量级线性时序注意力,以紧凑的键值状态增量吸收被移除的标记,实现远距离依赖的低开销保留;其次在局部滑动窗口中引入块稀疏注意力,减少短程建模中的冗余计算,将算力重分配至更关键依赖。最后,设计适配混合注意力的解耦蒸馏策略:先在密集注意力下进行少量步数的初始蒸馏,再激活线性时序与块稀疏注意力用于流式建模,保障优化稳定性。大量实验表明,Hybrid Forcing 在短/长视频生成基准上持续达到最先进性能。尤为关键的是,该模型在单张 NVIDIA H100 GPU 上实现无量化、无压缩的 832x480 视频实时生成,帧率高达 29.5 FPS。源代码与训练模型已公开于 https://github.com/leeruibin/hybrid-forcing。
原文摘要 · Abstract (English)
Streaming video generation (SVG) distills a pretrained bidirectional video diffusion model into an autoregressive model equipped with sliding window attention (SWA). However, SWA inevitably loses distant history during long video generation, and its computational overhead remains a critical challenge to real-time deployment. In this work, we propose Hybrid Forcing, which jointly optimizes temporal information retention and computational efficiency through a hybrid attention design. First, we introduce lightweight linear temporal attention to preserve long-range dependencies beyond the sliding window. In particular, we maintain a compact key-value state to incrementally absorb evicted tokens, retaining temporal context with negligible memory and computational overhead. Second, we incorporate block-sparse attention into the local sliding window to reduce redundant computation within short-range modeling, reallocating computational capacity toward more critical dependencies. Finally, we introduce a decoupled distillation strategy tailored to the hybrid attention design. A few-step initial distillation is performed under dense attention, then the distillation of our proposed linear temporal and block-sparse attention is activated for streaming modeling, ensuring stable optimization. Extensive experiments on both short- and long-form video generation benchmarks demonstrate that Hybrid Forcing consistently achieves state-of-the-art performance. Notably, our model achieves real-time, unbounded 832x480 video generation at 29.5 FPS on a single NVIDIA H100 GPU without quantization or model compression. The source code and trained models are available at https://github.com/leeruibin/hybrid-forcing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。