让长视频生成自动识别重要帧、跳过简单步骤,提升质量和效率。
Surprise Forcing: What to Remember, When to Skip in Long Video Generation

- 用惊喜度门控记忆库动态记住关键帧,避免遗忘。
- 根据画面变化程度智能跳过冗余去噪步骤,节省计算资源。
- 无需训练即可提升长视频连贯性与画质,适合实时生成场景。
流式自回归扩散模型使分钟级视频合成成为可能,但其有限上下文和固定去噪调度在高度非平稳序列中均匀分配资源。滚动的键值缓存会遗忘远端视觉证据,即使这些证据仍很重要;而每个生成块都接受相同次数的去噪迭代,无论实际难度如何。本文提出无需训练的Surprise Forcing框架,将上述问题视为在线资源分配任务。一个惊喜门控记忆库用价值令牌描述符总结被替换的帧,通过全局偏差与最近邻新颖性信号评估其重要性,并在归一化得分空间中通过反馈控制预算调节准入。基于优先级的替换与相关性感知路由使外部记忆保持紧凑且有效。同时,惊喜感知去噪通过首次去噪后相邻帧最大余弦距离估计块难度,使用局部百分位调度器跳过较易块的中间步骤。在VBench、VBench-Long和VBench-2.0上的实验表明,该策略提升了长时序一致性与视觉质量,同时保持实时流式吞吐率。
原文摘要 · Abstract (English)
Streaming autoregressive diffusion makes minute-scale video synthesis practical, but its bounded context and fixed denoising schedule allocate resources uniformly across a highly non-stationary sequence. A rolling key-value cache forgets distant visual evidence even when that evidence remains important, while every generated chunk receives the same number of denoising passes irrespective of its actual difficulty. We introduce Surprise Forcing, a training-free framework that treats both limitations as online resource-allocation problems. A Surprise-Gated Memory Bank summarizes evicted frames with value-token descriptors, evaluates them using complementary global-deviation and nearest-neighbor novelty signals, and regulates admission through a feedback-controlled budget in normalized score space. Priority-based replacement and relevance-aware routing then keep the external memory compact and useful. In parallel, Surprise-Aware Denoising estimates chunk difficulty from the maximum adjacent-frame cosine distance after the first denoising pass and uses a local percentile scheduler to skip intermediate steps for comparatively easy chunks. Experiments on VBench, VBench-Long, and VBench-2.0 show that the proposed allocation strategy improves long-horizon consistency and visual quality while retaining real-time streaming throughput.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。