将视频生成模型蒸馏为几步完成的快速生成器,提升实时应用效率。
Transition Matching Distillation for Fast Video Generation

- 通过匹配扩散模型的多步去噪轨迹,设计轻量级条件流进行少步生成
- 在1.3B和14B模型上实现速度与画质的优异平衡,少步生成效果领先
- 适合需要快速生成视频的交互式应用,如游戏、实时创作场景
大型视频扩散和光流模型在高质量视频生成方面取得显著进展,但其多步采样过程效率低下,限制了在实时交互应用中的使用。本文提出过渡匹配蒸馏(TMD),一种将视频扩散模型蒸馏为高效少步生成器的新框架。TMD的核心思想是将扩散模型的多步去噪轨迹匹配到少步概率转移过程,每一步由轻量级条件流建模。为实现高效蒸馏,将原始扩散主干分解为两部分:(1) 主干网络,包含大部分早期层,用于在每个外层转移步骤中提取语义表征;(2) 流头,由最后几层组成,利用这些表征执行多次内层流更新。给定预训练视频流模型后,首先引入流头并将其适配为条件流映射,然后在每一步转移中对带流头的学生模型进行分布匹配蒸馏。在蒸馏Wan2.1 1.3B和14B文本到视频模型上的大量实验表明,TMD在生成速度与视觉质量之间提供了灵活而强大的权衡。特别地,在相近推理成本下,TMD在视觉保真度和提示遵循性方面优于现有蒸馏模型。
原文摘要 · Abstract (English)
Large video diffusion and flow models have achieved remarkable success in high-quality video generation, but their use in real-time interactive applications remains limited due to their inefficient multi-step sampling process. In this work, we present Transition Matching Distillation (TMD), a novel framework for distilling video diffusion models into efficient few-step generators. The central idea of TMD is to match the multi-step denoising trajectory of a diffusion model with a few-step probability transition process, where each transition is modeled as a lightweight conditional flow. To enable efficient distillation, we decompose the original diffusion backbone into two components: (1) a main backbone, comprising the majority of early layers, that extracts semantic representations at each outer transition step; and (2) a flow head, consisting of the last few layers, that leverages these representations to perform multiple inner flow updates. Given a pretrained video flow model, we first introduce a flow head to the model, and adapt it into a conditional flow map. We then apply distribution matching distillation to the student model with flow head rollout in each transition step. Extensive experiments on distilling Wan2.1 1.3B and 14B text-to-video models demonstrate that TMD provides a flexible and strong trade-off between generation speed and visual quality. In particular, TMD outperforms existing distilled models under comparable inference costs in terms of visual fidelity and prompt adherence. Project page: https://research.nvidia.com/labs/genair/tmd
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。