arXiv:2506.03123cs.CV2025-06ICCV被引 20

通过双专家分工提升视频生成速度与质量,减少冗余计算。

Dual-Expert Consistency Model for Efficient and High-Quality Video Generation

  • 分设语义与细节专家,分别优化运动布局和精细纹理
  • 仅需8步采样即达当前最佳视觉效果,显著提速
  • 适合追求高效高质视频生成的研究者与开发者

扩散模型在视频合成中表现卓越,但需多次去噪步骤,计算开销大。一致性模型虽能加速扩散过程,但直接用于视频生成时易导致时间一致性下降和外观细节模糊。本文分析一致性模型训练动态,发现蒸馏过程中不同时间步的梯度差异与损失贡献不均,阻碍学生模型达到最优状态。为此提出参数高效的双专家一致性模型(DCM):语义专家专注学习语义布局与运动,细节专家负责精细纹理优化;引入时间连贯性损失增强语义专家的运动一致性,结合GAN与特征匹配损失提升细节专家的生成质量。实验表明,该方法在仅8步采样下实现当前最优视觉质量,验证了专家分工在视频扩散模型蒸馏中的有效性。代码与模型已开源。

原文摘要 · Abstract (English)

Diffusion Models have achieved remarkable results in video synthesis but require iterative denoising steps, leading to substantial computational overhead. Consistency Models have made significant progress in accelerating diffusion models. However, directly applying them to video diffusion models often results in severe degradation of temporal consistency and appearance details. In this paper, by analyzing the training dynamics of Consistency Models, we identify a key conflicting learning dynamics during the distillation process: there is a significant discrepancy in the optimization gradients and loss contributions across different timesteps. This discrepancy prevents the distilled student model from achieving an optimal state, leading to compromised temporal consistency and degraded appearance details. To address this issue, we propose a parameter-efficient \textbf{Dual-Expert Consistency Model~(DCM)}, where a semantic expert focuses on learning semantic layout and motion, while a detail expert specializes in fine detail refinement. Furthermore, we introduce Temporal Coherence Loss to improve motion consistency for the semantic expert and apply GAN and Feature Matching Loss to enhance the synthesis quality of the detail expert.Our approach achieves state-of-the-art visual quality with significantly reduced sampling steps, demonstrating the effectiveness of expert specialization in video diffusion model distillation. Our code and models are available at \href{https://github.com/Vchitect/DCM}{https://github.com/Vchitect/DCM}.

视频生成一致性模型扩散模型高效生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。