arXiv:2607.20940cs.CV2026-07

通过动态调整分块粒度,提升视频生成效率与质量。

Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention

论文配图:Ms. Forcing: Efficient Streaming Video Generation with Multi-Scale Patchification and Attention
图 1 · 摘自论文原文
  • 根据噪声水平自适应粗化分块,减少冗余计算
  • 单卡实测达22.84帧/秒,比基线快39.6%
  • 适合追求高帧率、长视频生成的实时应用

流式视频扩散模型在动态世界模拟中取得显著进展,但传统逐帧生成的嵌套自回归与去噪循环阻碍了实时部署。近期滚动窗口方法在多个连续帧上分层去噪,提升了吞吐量和长时稳定性。然而,它们对所有状态采用相同细粒度分块,导致联合去噪窗口中存在大量依赖噪声的冗余。我们提出Ms.Forcing,一种高效流式视频生成范式,根据各状态噪声水平自适应调整空间粒度。其多尺度分块(MSP)为更嘈杂状态分配更粗的分块,使活跃窗口令牌数减少45%;多尺度自注意力(MSSA)则匹配查询尺度的可见非汇点键值密度,进一步降低注意力开销。由于调度由窗口位置固定,Ms.Forcing保持静态且硬件友好的计算图。我们还引入同噪声级去噪记忆驱动(H-DMD),从共享相同源噪声级别的干净预测中合成假视频,从而降低训练序列与推理滚动间的不匹配。多尺度设计有助于抵消通过重叠窗口反向传播带来的额外训练成本。定量与定性实验表明,Ms.Forcing在单张H200 GPU上达到22.84 FPS,比Rolling Forcing快39.6%,并在短视频与长视频生成场景中显著提升VBench评分。

原文摘要 · Abstract (English)

Streaming video diffusion models have made substantial progress toward interactive and dynamic world simulation, but the nested autoregressive and denoising loops of conventional next-frame generation hinder real-time deployment. Recent rolling-window methods pipeline denoising across multiple consecutive frames at different noise levels, improving throughput and long-horizon stability. However, they tokenize every state at the same fine spatial granularity, leaving substantial noise-dependent redundancy in the joint denoising window. We propose Ms.Forcing, an efficient streaming video generation paradigm that adapts spatial granularity to each state's noise level. Its Multi-Scale Patchification (MSP) assigns coarser patches to noisier states, reducing the active-window token count by 45%, while Multi-Scale Self-Attention (MSSA) matches the density of visible non-sink keys and values to each query scale to further reduce attention cost. Because both schedules are fixed by window position, Ms.Forcing retains a static, hardware-friendly computation graph. We further introduce Homogeneous-Noise-Level DMD (H-DMD), which assembles each fake video from clean predictions sharing the same source noise level, thereby reducing the mismatch between DMD training sequences and inference-time rollouts. The multi-scale design helps offset the additional training cost of backpropagating through overlapping windows. We include both quantitative and qualitative experiments to show that Ms.Forcing reaches 22.84 FPS on a single H200 GPU, 39.6% faster than Rolling Forcing, while significantly improving VBench scores in both short video and long video generation setting.

视频生成扩散模型效率优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。