用图像质量评分替代逐帧采样,实现视频生成的快速推理。
Speculative Decoding for Autoregressive Video Generation

- 用图像质量评分器替代传统令牌验证,实现视频块的快速筛选。
- 在保持98.1%生成质量前提下,速度提升1.59倍,最高达2.09倍。
- 无需训练、不改架构,可直接接入现有视频生成流程。
自回归视频扩散正成为流式视频合成的有前途范式,步骤蒸馏是加速推理的主要手段。但自回归视频生成中视频块为连续时空张量,缺乏逐标记分布,使得主流大语言模型的推测解码难以直接适用。本文提出SDVG,通过将标记验证替换为图像质量路由,首次将推测解码引入基于块的自回归视频扩散。1.3B的草稿模型经4步去噪生成候选块;每个块经VAE解码后由ImageReward评分,采用最差帧聚合(取每帧奖励最小值)以捕捉平均化会掩盖的单帧伪影。得分高于固定阈值tau的块被纳入14B目标模型的KV缓存,其余由目标模型重生成。两个关键设计:首块强制拒绝以锚定场景结构,tau作为单一调节旋钮,可平滑控制质量-速度权衡。在1003个MovieGenVideoBench提示(832x480)上,τ=-0.7时,仅损失1.9%质量(VisionReward: 0.0773 vs. 0.0788),获得1.59倍加速;2.09倍加速下仍保留95.7%质量,且始终优于仅用草稿生成超17%。该框架无需训练、无需架构改动,可无缝集成至现有自回归视频生成流水线。
原文摘要 · Abstract (English)
Autoregressive video diffusion is emerging as a promising paradigm for streaming video synthesis, with step distillation serving as the primary means of accelerating inference. Whether speculative decoding, the dominant acceleration strategy for large language models, can be effectively adapted to autoregressive video generation remains an open question, because video blocks are continuous spatiotemporal tensors with no token-level distribution for exact rejection sampling. We introduce SDVG, which brings speculative decoding to block-based autoregressive video diffusion by replacing token verification with an image-quality router. A 1.3B drafter proposes candidate blocks via four denoising steps; each block is VAE-decoded and scored by ImageReward using worst-frame aggregation--taking the minimum per-frame reward to catch single-frame artifacts that averaging would mask. Blocks scoring above a fixed threshold tau are accepted into the 14B target's KV cache; the rest are regenerated by the target. Two additional design choices prove critical: the first block is always force-rejected to anchor scene composition, and tau serves as a single knob that traces a smooth quality-speed Pareto frontier. On 1003 MovieGenVideoBench prompts (832x480), SDVG retains 98.1% of target-only VisionReward quality (0.0773 vs. 0.0788) at a 1.59x speedup with tau=-0.7, and reaches 2.09x at 95.7% quality retention--while consistently outperforming draft-only generation by over +17%. The framework is training-free, requires no architectural changes, and can be seamlessly integrated into existing autoregressive video generation pipelines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。