提出SGMD方法,让视频扩散模型蒸馏更快更流畅。
SGMD: Score Gradient Matching Distillation for Few-Step Video Diffusion Distillation

- 用伪得分直接优化教师模型,稳定匹配分布
- 训练速度提升约3倍,4步蒸馏模型运动更自然
- 适合需要快速生成高质量视频的研究与应用
分布匹配蒸馏(DMD)是加速少步视频扩散模型推理的常用方法。然而,DMD式视频蒸馏面临两个耦合挑战:伪得分需持续追踪不断演化的生成器,频繁更新导致训练成本高;而反向KL式匹配易陷入模式,难以保留强运动动态。为此,我们提出**得分梯度匹配蒸馏(SGMD)**。SGMD采用伪得分视角,直接优化伪得分以逼近教师模型,并利用教师停止梯度的Fisher信息作为稳定分布匹配目标。通过梯度分析,我们证明该目标在理想追踪下具有理论依据。基于此,SGMD引入一对对偶势能:负残差(NR)用于外循环校正,残差收缩(RC)用于内循环追踪。实验表明,相比DMD2,SGMD实现约3倍训练速度提升,显著改善4步蒸馏模型的运动动态,同时保持时间一致性。人类评估显示,SGMD在运动质量与整体偏好上更受青睐,视觉质量和文本对齐性能相当。代码已公开于https://github.com/ModelTC/LightX2V。
原文摘要 · Abstract (English)
Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models. However, DMD-style video distillation faces two coupled challenges: the fake score must track a continuously evolving generator, making training costly when frequent updates are required, while reverse-KL-style matching can be mode-seeking and conservative for preserving strong motion dynamics. To address these issues, we propose \textbf{Score Gradient Matching Distillation (SGMD)}. SGMD adopts a fake-score perspective by directly optimizing the fake score toward the teacher, while using teacher stop-gradient Fisher as a stable distribution-matching objective. We provide a gradient analysis that motivates this objective choice under ideal tracking. Building on this, SGMD introduces a pair of dual potentials: negative-residual (NR) for outer-loop correction and residual-contraction (RC) for inner-loop tracking. Empirically, compared to DMD2, SGMD achieves an approximately $\sim 3\times$ training speedup and substantially improves motion dynamics for 4-step distilled models while preserving temporal consistency. A human study confirms that SGMD is preferred in motion quality and overall preference, while visual quality and text alignment remain comparable. Code is available at https://github.com/ModelTC/LightX2V.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。