提升视频生成蒸馏速度,解决快速推理时的运动不连贯问题。
CoDMD: Copula-aware Distribution Matching Distillation for Fast Video Generation

- 引入基于配对关系的轻量级正则化,约束样本间与帧间的几何结构。
- 在4步内完成50步教师模型的蒸馏,速度提升25倍,质量超越现有方法。
- 无需额外网络或数据,适合作为视频生成加速的通用解决方案。
少步蒸馏在视频扩散模型中受到广泛关注,以满足实际场景对高效部署的需求。然而,主流的分布匹配蒸馏(DMD)在有限的NFE预算下表现下降,导致视频生成出现布局不稳定、过饱和和运动动态断裂等问题。我们发现其根源在于:标准DMD是逐样本的分布匹配目标,依赖坐标梯度,未显式约束批量元素或时间帧间的关联几何结构,使底层的配对结构(copula)缺乏控制。结合反KL目标的模式寻找倾向,这种缺乏关系引导的特性使其在少步设置下易陷入局部最优。为此,我们提出配对感知的分布匹配蒸馏(CoDMD),利用冻结教师模型与在线伪模型已生成的得分估计,构建跨样本与跨帧的成对关系矩阵,并通过额外的分布目标进行匹配,无需额外网络、数据集或采样轨迹。在Wan-2.1-T2V模型系列(1.3B & 14B规模)上,将50步教师蒸馏为4步学生模型,实现约25×加速,获得VBench得分84.46与84.87,优于先前基于轨迹的方法(rCM 82.81 & 84.05)和基于分布的方法(DMD 83.38 & 83.81)。
原文摘要 · Abstract (English)
Few-step distillation for video diffusion models has attracted significant attention, driven by the urgent demand for efficient deployment in real-world scenarios. However, Distribution Matching Distillation (DMD), a leading paradigm, tends to degrade under limited NFE budgets, manifesting in video generation as layout instability, oversaturation, and broken motion dynamics. We trace this failure to a structural limitation: standard DMD is an intra-sample distribution-matching objective with coordinate-wise gradients, and thus imposes no explicit constraint on the relational geometry across batch elements or temporal frames, leaving the underlying copula largely unregulated. Combined with the mode-seeking tendency of its reverse-KL objective, this absence of relational guidance makes DMD prone to collapsing into local optima in the few-step regime. Motivated by this insight, we propose Copula-aware DMD (CoDMD), a lightweight relational regularizer that reuses score estimates already produced by the frozen teacher and the online fake model to construct pairwise relation matrices across samples and frames. These are matched through a supplementary distributional objective that requires no additional networks, datasets, or sampling trajectories. On the Wan-2.1-T2V model series at 1.3B & 14B scales, CoDMD distills 50-step teachers into 4-step students, achieving an approximate 25$\times$ speed-up while attaining VBench scores of 84.46 & 84.87, outperforming prior trajectory-based (rCM 82.81 & 84.05) and distribution-based (DMD 83.38 & 83.81) methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。