无需训练即可加速视频生成，让扩散模型推理快8倍以上
Training-Free Efficient Video Generation via Dynamic Token Carving
- 动态选择关键帧交互，用3D空间填充曲线优化注意力计算
- 分阶段提升图像分辨率，早期用低分辨率减少计算量
- 插件式部署，不需重训练，推理时间从分钟级缩至秒级
尽管视频扩散变换器（DiT）模型生成质量出色，但其实际部署受限于巨大的计算开销。这主要源于自注意力机制随序列长度呈二次增长，以及扩散模型的多步生成特性。为此，我们提出Jenga——一种结合动态注意力剪枝与渐进分辨率生成的新推理流程。核心思路是：早期去噪步骤无需高分辨率潜在表示，后期步骤无需密集注意力。Jenga采用块级注意力机制，利用3D空间填充曲线动态筛选相关特征交互，并通过逐步提升潜空间分辨率实现高效生成。实验表明，Jenga在多个领先视频扩散模型上均实现显著加速（在VBench上达8.83倍提速，性能仅下降0.01%），且作为即插即用方案，可在现代硬件上将推理时间从分钟级压缩至秒级，无需模型重训练。
原文摘要 · Abstract (English)
Despite the remarkable generation quality of video Diffusion Transformer (DiT) models, their practical deployment is severely hindered by extensive computational requirements. This inefficiency stems from two key challenges: the quadratic complexity of self-attention with respect to token length and the multi-step nature of diffusion models. To address these limitations, we present Jenga, a novel inference pipeline that combines dynamic attention carving with progressive resolution generation. Our approach leverages two key insights: (1) early denoising steps do not require high-resolution latents, and (2) later steps do not require dense attention. Jenga introduces a block-wise attention mechanism that dynamically selects relevant token interactions using 3D space-filling curves, alongside a progressive resolution strategy that gradually increases latent resolution during generation. Experimental results demonstrate that Jenga achieves substantial speedups across multiple state-of-the-art video diffusion models while maintaining comparable generation quality (8.83$\times$ speedup with 0.01\% performance drop on VBench). As a plug-and-play solution, Jenga enables practical, high-quality video generation on modern hardware by reducing inference time from minutes to seconds -- without requiring model retraining. Code: https://github.com/dvlab-research/Jenga
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。