arXiv:2509.22622cs.CV2025-09被引 212

实现秒级生成超长视频,支持实时交互提示。

LongLive: Real-time Interactive Long Video Generation

  • 帧级自回归架构+缓存刷新机制,保障长视频生成效率与质量。
  • 单卡H100下达20.7帧/秒,支持最长240秒视频生成。
  • 支持实时输入提示,适合动态创作场景的开发者和内容创作者。

我们提出LongLive,一种面向实时交互式长视频生成的帧级自回归(AR)框架。长视频生成在效率与质量上面临挑战:扩散模型虽质量高但效率低,因果注意力自回归模型推理快却因训练记忆负担导致长视频质量下降。此外,除静态提示外,实时流式提示输入对动态内容创作至关重要,但需保证提示切换时视觉一致性和语义连贯性。LongLive采用因果帧级自回归设计,结合KV缓存重刷新机制,使提示切换更平滑;引入流式长序列微调,实现训练与推理一致(train-long-test-long);并使用短窗口注意力与帧级注意力下沉(frame sink),在加速生成的同时保持长程一致性。通过这些设计,LongLive仅用32个GPU天即微调1.3B参数短片段模型,实现分钟级长视频生成。推理阶段,单张NVIDIA H100可达20.7 FPS,在VBench评测中短/长视频表现均优异,支持最高240秒视频生成,并可实现INT8量化推理,仅轻微损失质量。

原文摘要 · Abstract (English)

We present LongLive, a frame-level autoregressive (AR) framework for real-time and interactive long video generation. Long video generation presents challenges in both efficiency and quality. Diffusion and Diffusion-Forcing models can produce high-quality videos but suffer from low efficiency due to bidirectional attention. Causal attention AR models support KV caching for faster inference, but often degrade in quality on long videos due to memory challenges during long-video training. In addition, beyond static prompt-based generation, interactive capabilities, such as streaming prompt inputs, are critical for dynamic content creation, enabling users to guide narratives in real time. This interactive requirement significantly increases complexity, especially in ensuring visual consistency and semantic coherence during prompt transitions. To address these challenges, LongLive adopts a causal, frame-level AR design that integrates a KV-recache mechanism that refreshes cached states with new prompts for smooth, adherent switches; streaming long tuning to enable long video training and to align training and inference (train-long-test-long); and short window attention paired with a frame-level attention sink, shorten as frame sink, preserving long-range consistency while enabling faster generation. With these key designs, LongLive fine-tunes a 1.3B-parameter short-clip model to minute-long generation in just 32 GPU-days. At inference, LongLive sustains 20.7 FPS on a single NVIDIA H100, achieves strong performance on VBench in both short and long videos. LongLive supports up to 240-second videos on a single H100 GPU. LongLive further supports INT8-quantized inference with only marginal quality loss.

长视频生成自回归实时交互高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。