通过频域锚定解决长视频生成中的误差累积问题。
FreqForcing: Autoregressive Long Video Generation via Spectral Self-Anchoring
- 利用低频成分锚定视觉稳定性，保留高频动态运动
- 实现2分钟视频生成，相比5秒预训练扩展24倍
- 无需训练即可显著提升长视频质量，适合实时生成场景
自回归视频扩散模型支持实时流式生成，但自滚动过程中误差随时间累积，表现为色彩漂移、运动停滞甚至视觉崩溃。本文从频域角度分析发现，误差积累体现为低频带能量显著漂移。尽管频域注意力抑制器能部分缓解该问题，仍无法彻底解决。为此，我们提出FregForcing框架，采用频域自锚定（SSA）机制：利用锚定注意力的低频成分维持长时程视觉稳定，同时通过局部注意力的高频成分保留动态运动。该方法将预训练于5秒片段的模型扩展至2分钟生成，实现24倍外推。大量实验表明，FreqForcing在定量与定性指标上均优于现有无训练方法，且媲美代表性有训练方法。
原文摘要 · Abstract (English)
Autoregressive video diffusion models enable real-time streaming video generation. However, errors introduced during self-rollout accumulate over long horizons, manifesting as color drift, motion stagnation, and eventual visual collapse. In this paper, we characterize this phenomenon from a frequency-domain perspective: error accumulation appears as a pronounced energy drift in the low-frequency bands. We further investigate the effectiveness of attention sink in the frequency domain, and find that it improves the video quality by alleviating the spectral energy drift to some extent, but cannot fully resolve it. Motivated by the above analysis, we propose FreqForcing, a training-free framework that addresses error accumulation in long-video generation via Spectral Self-Anchoring (SSA). The proposed SSA leverages the low-frequency components of anchor attention to maintain long-horizon visual stability, while preserving dynamic motion through the high-frequency components of local attention. Our FreqForcing extends Self-Forcing pretrained on 5s clips to two-minute generation, achieving 24x extrapolation. Extensive experiments show that FreqForcing outperforms existing training-free methods quantitatively and qualitatively while remaining competitive with representative training-based approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。