让视频生成模型按需调整计算量,提速30倍且画质不降。
Dynamic-in-Few-Step: Unifying Dynamic Computation and Few-Step Distillation for Efficient Video Generation

- 在少步蒸馏中动态调整每步计算结构,避免固定架构浪费资源。
- 在Wan-14B上比4步蒸馏再减24%算力,实现30倍速度提升。
- 适合追求高效视频生成的开发者和部署团队。
视频扩散模型(VDMs)虽生成质量高,但计算成本巨大。现有少步蒸馏技术虽能加速推理,但通常对所有去噪阶段使用静态模型结构,忽略了不同噪声水平下的实际计算需求差异。本文提出一种后训练加速框架,将动态结构稀疏化直接融入蒸馏过程,联合优化去噪步骤与模型稀疏性,将预训练的VDM转化为针对每一步的混合模型(MoM)。为解决联合优化带来的训练不稳定性,引入渐进式训练策略与输出滚动机制,确保时间步间结构决策的一致性。同时开发专用推理引擎以高效部署该MoM。本方法与现有加速技术正交且高效:在Wan-14B数据集上,在4步蒸馏基础上额外减少24%每步浮点运算量,带来1.2倍实际运行速度提升,相较50步教师模型实现30倍加速,同时保持优异生成质量。
原文摘要 · Abstract (English)
Video Diffusion Models (VDMs) have demonstrated superior generation quality but suffer from prohibitive computational costs. While recent few-step distillation techniques significantly accelerate inference, they typically enforce a static model architecture across all denoising stages, ignoring the varying computational demands inherent to different noise levels. In this work, we propose a novel post-training acceleration framework that exploits this redundancy by integrating dynamic structural sparsification directly into the distillation process. Unlike conventional post-hoc compression applied to a fixed diffusion pipeline, our approach jointly optimizes the denoising steps and structured model sparsity, transforming a pre-trained VDM into a compact, step-specific Mixture-of-Models (MoM). To address the training instability arising from this joint optimization, we introduce a Progressive Training Strategy coupled with an Output Rollout Mechanism, which ensures the coherent learning of structural decisions across timesteps. Furthermore, we develop a specialized inference engine to deploy the resulting MoM efficiently. Our method is orthogonal to existing acceleration techniques and highly effective: On Wan-14B, it removes 24% of the per-step FLOPs on top of 4-step distillation, adding a 1.2x wall-clock gain and reaching a 30x speedup over the 50-step teacher while preserving competitive generation quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。