arXiv:2511.07399cs.CVcs.LG2025-11中稿 · MLSys 2026被引 25

StreamDiffusionV2实现低延迟视频生成,支持实时互动直播。

StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation

  • 采用SLO感知批处理与块调度,保障帧间延迟稳定
  • 首帧生成<0.5秒,14B模型达58.28 FPS,支持灵活采样步数
  • 无需量化即可实现高吞吐,适合个人到企业级部署

生成模型正在重塑直播行业,重新定义内容创作、风格化与分发方式。此前基于图像的流式扩散模型虽高效创意,但受限于图像基础设计,在时序一致性上表现不足。近期视频扩散模型显著提升了时序一致性和采样效率,但离线系统主要通过批量处理优化吞吐量。相比之下,实时在线直播需满足严格服务级别目标(SLO):首帧时间极短,每帧必须在单帧截止时间内完成且抖动低。此外,多GPU实时流的可扩展性尚未解决。为此,我们提出StreamDiffusionV2,一种无需训练的交互式直播视频生成流水线,集成SLO感知批处理调度器与块调度器,结合基于汇流令牌的滚动KV缓存、运动感知噪声控制器等系统优化。进一步引入可扩展的流水线编排,跨去噪步骤与网络层并行化,实现近线性帧率提升而不违反延迟约束。系统可在异构GPU环境无缝扩展,支持1–4步去噪,兼顾超低延迟与高质量模式。无需TensorRT或量化,4张H100 GPU上,14B参数模型实现58.28 FPS,1.3B模型达64.52 FPS,使前沿生成式直播成为现实,适用于从创作者到企业级平台的广泛场景。

原文摘要 · Abstract (English)

Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powered efficient and creative live streaming products but have hit limits on temporal consistency due to the foundation of image-based designs. Recent advances in video diffusion have markedly improved temporal consistency and sampling efficiency for offline generation. However, offline generation systems primarily optimize throughput by batching large workloads. In contrast, live online streaming operates under strict service-level objectives (SLOs): time-to-first-frame must be minimal, and every frame must meet a per-frame deadline with low jitter. Besides, scalable multi-GPU serving for real-time streams remains largely unresolved so far. To address this, we present StreamDiffusionV2, a training-free pipeline for interactive live streaming with video diffusion models. StreamDiffusionV2 integrates an SLO-aware batching scheduler and a block scheduler, together with a sink-token--guided rolling KV cache, a motion-aware noise controller, and other system-level optimizations. Moreover, we introduce a scalable pipeline orchestration that parallelizes the diffusion process across denoising steps and network layers, achieving near-linear FPS scaling without violating latency guarantees. The system scales seamlessly across heterogeneous GPU environments and supports flexible denoising steps (e.g., 1--4), enabling both ultra-low-latency and higher-quality modes. Without TensorRT or quantization, StreamDiffusionV2 renders the first frame within 0.5s and attains 58.28 FPS with a 14B-parameter model and 64.52 FPS with a 1.3B-parameter model on four H100 GPUs, making state-of-the-art generative live streaming practical and accessible--from individual creators to enterprise-scale platforms.

视频生成实时渲染扩散模型流式系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。