通过动态更新的上下文记忆和奖励驱动的分布蒸馏,实现高效高质量视频流生成。
Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation
- 用指数移动平均更新滑动窗口中的上下文记忆,避免初始帧重复。
- 基于视觉语言模型评分优先蒸馏动态内容,显著提升运动质量。
- 在单张H100上实现23.1帧/秒的实时生成,适合交互式应用。
高效视频流生成对模拟交互式动态世界至关重要。现有方法使用滑动窗口注意力蒸馏少步数视频扩散模型,以初始帧作为固定记忆点维持注意力性能并减少误差累积。然而,视频帧过度依赖这些静态记忆点,导致初始帧被重复复制,运动动态性下降。为此,我们提出奖励强制(Reward Forcing)框架,包含两项关键设计:首先,提出EMA-Sink,通过指数移动平均融合退出窗口的旧记忆点,持续更新固定大小的记忆令牌,无需额外计算开销即可捕捉长期上下文与近期动态,防止初始帧重复并保持长时一致性。其次,为更好蒸馏教师模型中的运动动态,提出新颖的奖励分布匹配蒸馏(Re-DMD)。传统分布匹配对所有样本同等处理,限制模型对动态内容的优先学习能力;而Re-DMD通过视觉语言模型对样本动态性评分,引导模型输出分布偏向高奖励区域,从而显著提升运动质量同时保持数据保真度。我们在多个标准基准上进行了定量与定性实验,结果表明,Reward Forcing在保持最先进性能的同时,可在单张H100 GPU上实现23.1帧/秒的高质量视频流生成。
原文摘要 · Abstract (English)
Efficient streaming video generation is critical for simulating interactive and dynamic worlds. Existing methods distill few-step video diffusion models with sliding window attention, using initial frames as sink tokens to maintain attention performance and reduce error accumulation. However, video frames become overly dependent on these static tokens, resulting in copied initial frames and diminished motion dynamics. To address this, we introduce Reward Forcing, a novel framework with two key designs. First, we propose EMA-Sink, which maintains fixed-size tokens initialized from initial frames and continuously updated by fusing evicted tokens via exponential moving average as they exit the sliding window. Without additional computation cost, EMA-Sink tokens capture both long-term context and recent dynamics, preventing initial frame copying while maintaining long-horizon consistency. Second, to better distill motion dynamics from teacher models, we propose a novel Rewarded Distribution Matching Distillation (Re-DMD). Vanilla distribution matching treats every training sample equally, limiting the model's ability to prioritize dynamic content. Instead, Re-DMD biases the model's output distribution toward high-reward regions by prioritizing samples with greater dynamics rated by a vision-language model. Re-DMD significantly enhances motion quality while preserving data fidelity. We include both quantitative and qualitative experiments to show that Reward Forcing achieves state-of-the-art performance on standard benchmarks while enabling high-quality streaming video generation at 23.1 FPS on a single H100 GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。