arXiv:2512.05754cs.CV2025-12被引 1

统一稀疏化让视频扩散模型加速83%,还保持画质。

USV: Unified Sparsification for Accelerating Video Diffusion Models

  • 联合优化注意力稀疏与采样步数,动态剪枝冗余计算。
  • 在大模型上实现83.3%去噪速度提升,端到端加速22.7%。
  • 适合需要高效高质视频生成的研究者与开发者。

高质量视频扩散模型(VDMs)的可扩展性受限于两大冗余来源:全局时空注意力的二次复杂度和长迭代去噪轨迹的计算开销。现有加速方法如稀疏注意力与步数蒸馏通常仅针对单一维度,难以持续提效。本文提出USV(Unified Sparsification for Video Diffusion Models),一个端到端可训练框架,通过联合调控模型内部计算与采样过程的稀疏化,突破此限制。USV学习一种依赖数据与时间步的动态稀疏策略,剪枝冗余注意力连接,自适应合并语义相似的视觉令牌,并减少去噪步数,将这些操作视为同一优化目标下的协同动作。这种多维协同设计实现了此前分离策略间的强互补增益。大规模视频生成基准测试表明,USV在去噪阶段最高提速83.3%,端到端加速达22.7%,同时保持高视觉保真度。结果凸显统一动态稀疏化是实现高效高质量视频生成的可行路径。

原文摘要 · Abstract (English)

The scalability of high-fidelity video diffusion models (VDMs) is constrained by two key sources of redundancy: the quadratic complexity of global spatio-temporal attention and the computational overhead of long iterative denoising trajectories. Existing accelerators -- such as sparse attention and step-distilled samplers -- typically target a single dimension in isolation and quickly encounter diminishing returns, as the remaining bottlenecks become dominant. In this work, we introduce USV (Unified Sparsification for Video diffusion models), an end-to-end trainable framework that overcomes this limitation by jointly orchestrating sparsification across both the model's internal computation and its sampling process. USV learns a dynamic, data- and timestep-dependent sparsification policy that prunes redundant attention connections, adaptively merges semantically similar tokens, and reduces denoising steps, treating them not as independent tricks but as coordinated actions within a single optimization objective. This multi-dimensional co-design enables strong mutual reinforcement among previously disjoint acceleration strategies. Extensive experiments on large-scale video generation benchmarks demonstrate that USV achieves up to 83.3% speedup in the denoising process and 22.7% end-to-end acceleration, while maintaining high visual fidelity. Our results highlight unified, dynamic sparsification as a practical path toward efficient, high-quality video generation.

视频生成扩散模型加速稀疏化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。