首个支持长视频生成的NVFP4全流程并行系统,显著提速降存。
LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

- 采用序列并行自回归训练,结合高效教师强制布局与分块VAE编码。
- 训练速度最高提升2.15倍,推理达45.7帧/秒,支持实时生成。
- 专为长视频设计,适合需要高效生成的科研与工业场景。
我们提出LongLive-2.0,一个基于NVFP4的全流程并行基础设施,用于长视频生成的训练与推理,解决速度与内存瓶颈。训练阶段引入序列并行自回归(AR)训练,实现平衡序列并行(Balanced SP),通过在每个计算节点上配对干净历史与噪声目标时间块,自然构建教师强制掩码,并支持SP感知的分块VAE编码。结合NVFP4精度,降低训练时的GPU内存占用并加速GEMM计算,随视频长度增加优势更明显。该系统无需依赖ODE初始化和分布匹配蒸馏(DMD),直接将扩散模型转化为长视频、多镜头、交互式自回归扩散模型,可进一步通过独立LoRA权重转为实时生成(4至2步去噪)。在Blackwell GPU上,支持W4A4 NVFP4推理,量化KV缓存节省内存,并通过异步流式VAE解码提升端到端吞吐。非Blackwell架构下部署序列并行推理以匹配黑堡性能,量化KV缓存减少跨GPU通信。实验显示训练最高提速2.15倍,推理提速1.84倍。LongLive-2.0-5B实现45.7 FPS推理,同时保持优异基准表现。据我们所知,LongLive-2.0是首个支持长视频生成的NVFP4训练与推理系统。
原文摘要 · Abstract (English)
We present LongLive-2.0, an NVFP4-based parallel infrastructure throughout the full training and inference workflow of long video generation, addressing speed and memory bottlenecks. For training, we introduce sequence-parallel autoregressive (AR) training, instantiated as Balanced SP, which co-designs the efficient teacher-forcing layout with SP execution by pairing clean-history and noisy-target temporal chunks on each rank, enabling a natural teacher-forcing mask with SP-aware chunked VAE encoding. Combined with NVFP4 precision, it reduces GPU memory cost and accelerates GEMM computation during training, the proportion of which increases as video length grows. Moreover, we show that a high-quality infrastructure and dataset enable a remarkably clean training pipeline. Unlike existing Self-Forcing series methods that rely on ODE initialization and subsequent distribution matching distillation (DMD), LongLive-2.0 directly tunes a diffusion model into a long, multi-shot, interactive auto-regressive (AR) diffusion model. It can be further converted to real-time generation (4 to 2 denoising steps) with standalone LoRA weights. For inference on Blackwell GPUs, we enable W4A4 NVFP4 inference, quantize KV cache into NVFP4 for memory savings, and boost end-to-end throughput with asynchronous streaming VAE decoding. On non-Blackwell GPU architectures, we deploy SP inference to match the speed on Blackwell GPUs, while the quantized KV cache can lower inter-GPU communication of SP. Experiments show up to 2.15x speedup in training, and 1.84x in inference. LongLive-2.0-5B achieves 45.7 FPS inference while attaining strong performance on benchmarks. To our knowledge, LongLive-2.0 is the first NVFP4 training and inference system for long video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。