arXiv:2511.20123cs.CV2025-11被引 13

提出无需训练的UltraViCo,突破视频生成长度限制至4倍。

UltraViCo: Breaking Extrapolation Limits in Video Diffusion Transformers

  • 通过抑制超长视频中远端令牌的注意力,解决注意力分散问题。
  • 在4倍外推下,图像质量提升40.5%,动态度提高233%。
  • 插件式设计,适用于多种模型和可控视频生成任务。

尽管取得进展,视频扩散变换器在超出训练长度时仍难以泛化,这一挑战被称为视频长度外推。我们识别出两种失效模式:模型特定的周期性内容重复和普遍的质量退化。先前工作仅通过位置编码缓解重复问题,忽视了质量退化,且外推能力有限。本文从更根本的角度重新审视该问题:注意力图直接决定上下文对输出的影响。我们发现两种失效模式源于同一原因——注意力分散,即训练窗口之外的令牌稀释了已学习的注意力模式。这导致质量下降,而重复是当这种分散形成周期性注意力模式时的特例,由位置编码的谐波特性引发。基于此洞察,我们提出UltraViCo,一种无需训练、可即插即用的方法,通过恒定衰减因子抑制训练窗口外令牌的注意力。该方法同时解决两类失效,显著优于多种基线模型,在不同模型和外推比例下均表现优异,将外推极限从2倍提升至4倍。尤其在4倍外推下,相比此前最佳方法,动态度与成像质量分别提升233%和40.5%。此外,该方法可无缝推广至可控视频合成与编辑等下游任务。

原文摘要 · Abstract (English)

Despite advances, video diffusion transformers still struggle to generalize beyond their training length, a challenge we term video length extrapolation. We identify two failure modes: model-specific periodic content repetition and a universal quality degradation. Prior works attempt to solve repetition via positional encodings, overlooking quality degradation and achieving only limited extrapolation. In this paper, we revisit this challenge from a more fundamental view: attention maps, which directly govern how context influences outputs. We identify that both failure modes arise from a unified cause: attention dispersion, where tokens beyond the training window dilute learned attention patterns. This leads to quality degradation and repetition emerges as a special case when this dispersion becomes structured into periodic attention patterns, induced by harmonic properties of positional encodings. Building on this insight, we propose UltraViCo, a training-free, plug-and-play method that suppresses attention for tokens beyond the training window via a constant decay factor. By jointly addressing both failure modes, we outperform a broad set of baselines largely across models and extrapolation ratios, pushing the extrapolation limit from 2x to 4x. Remarkably, it improves Dynamic Degree and Imaging Quality by 233% and 40.5% over the previous best method at 4x extrapolation. Furthermore, our method generalizes seamlessly to downstream tasks such as controllable video synthesis and editing.

视频生成扩散模型外推

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。