arXiv:2510.08431cs.CVcs.LG2025-10被引 64

提出rCM模型,让扩散模型生成更快更清晰,1-4步出高质量图像视频。

Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency

  • 用得分正则化改进连续时间一致性模型,解决细节生成差的问题。
  • 在140亿参数模型和5秒视频上,生成质量媲美顶尖方法,多样性更好。
  • 无需调GAN或大量参数,采样速度提升15到50倍,适合大规模应用。

尽管连续时间一致性模型(如sCM、MeanFlow)在理论和实践上对快速学术级扩散模型表现优异,但其在大规模文本到图像和视频任务中的应用仍受制于雅可比向量积(JVP)计算的基础设施挑战及评估基准(如FID)的局限性。本文首次将连续时间一致性模型扩展至通用应用级图像与视频扩散模型,并实现基于JVP的蒸馏在大规模下的有效应用。我们开发了兼容并行的FlashAttention-2 JVP内核,使sCM可在超过100亿参数的模型及高维视频任务上训练。研究发现sCM在精细细节生成上存在根本性质量问题,归因于误差累积及其前向发散目标的“模式覆盖”特性。为此,我们提出得分正则化的连续时间一致性模型(rCM),引入得分蒸馏作为长跳连接正则项。该设计结合了“模式寻找”式的反向发散,显著提升视觉质量同时保持高多样性。在高达140亿参数的Cosmos-Predict2和Wan2.1模型以及5秒视频上验证,rCM在质量指标上普遍达到当前最优蒸馏方法DMD2水平,有效缓解模式崩溃,且在多样性方面表现突出,全程无需使用GAN或复杂超参数调优。蒸馏模型仅需1~4步即可生成高保真样本,扩散采样速度提升15×~50×。结果表明,rCM是推进大规模扩散蒸馏的实用且理论扎实的框架。代码已公开于https://github.com/NVlabs/rcm。

原文摘要 · Abstract (English)

Although continuous-time consistency models (e.g., sCM, MeanFlow) are theoretically principled and empirically powerful for fast academic-scale diffusion, its applicability to large-scale text-to-image and video tasks remains unclear due to infrastructure challenges in Jacobian-vector product (JVP) computation and the limitations of evaluation benchmarks like FID. This work represents the first effort to scale up continuous-time consistency to general application-level image and video diffusion models, and to make JVP-based distillation effective at large scale. We first develop a parallelism-compatible FlashAttention-2 JVP kernel, enabling sCM training on models with over 10 billion parameters and high-dimensional video tasks. Our investigation reveals fundamental quality limitations of sCM in fine-detail generation, which we attribute to error accumulation and the "mode-covering" nature of its forward-divergence objective. To remedy this, we propose the score-regularized continuous-time consistency model (rCM), which incorporates score distillation as a long-skip regularizer. This integration complements sCM with the "mode-seeking" reverse divergence, effectively improving visual quality while maintaining high generation diversity. Validated on large-scale models (Cosmos-Predict2, Wan2.1) up to 14B parameters and 5-second videos, rCM generally matches the state-of-the-art distillation method DMD2 on quality metrics while mitigating mode collapse and offering notable advantages in diversity, all without GAN tuning or extensive hyperparameter searches. The distilled models generate high-fidelity samples in only $1\sim4$ steps, accelerating diffusion sampling by $15\times\sim50\times$. These results position rCM as a practical and theoretically grounded framework for advancing large-scale diffusion distillation. Code is available at https://github.com/NVlabs/rcm.

扩散模型视频生成蒸馏加速采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。