用高效蒸馏与联合强化学习,实现快速高保真图像生成。
Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning

- 提出时间步感知蒸馏策略,训练成本仅需原方法2.1%
- 联合蒸馏与强化学习,稳定训练并避免策略崩溃
- 少步采样下质量领先,适合追求效率与画质的场景
扩散模型虽为生成模型主流,但其迭代采样过程仍计算昂贵。时间步蒸馏可加速生成,但常需大量训练且导致画质下降。此外,使用强化学习(RL)对蒸馏模型进行美学或用户偏好微调时,易出现奖励滥用且训练不稳定。本文提出Flash-DMD框架,实现蒸馏加速与联合强化学习精炼。首先,设计一种高效的时间步感知蒸馏策略,显著降低训练成本,仅需DMD2的2.1%训练开销,同时提升真实感。其次,提出联合训练机制:在继续时间步蒸馏的同时,以强化学习目标微调模型。我们发现持续蒸馏提供的稳定、明确的损失项可作为强大正则化器,有效抑制强化学习训练中的崩溃问题。在基于得分和流匹配的模型上实验表明,Flash-DMD不仅收敛更快,且在少步采样下达到当前最优生成质量,在视觉质量、人类偏好及文本-图像对齐指标上均超越现有方法。本工作为训练高效、高保真且稳定的生成模型提供有效范式。代码即将发布。
原文摘要 · Abstract (English)
Diffusion Models have emerged as a leading class of generative models, yet their iterative sampling process remains computationally expensive. Timestep distillation is a promising technique to accelerate generation, but it often requires extensive training and leads to image quality degradation. Furthermore, fine-tuning these distilled models for specific objectives, such as aesthetic appeal or user preference, using Reinforcement Learning (RL) is notoriously unstable and easily falls into reward hacking. In this work, we introduce Flash-DMD, a novel framework that enables fast convergence with distillation and joint RL-based refinement. Specifically, we first propose an efficient timestep-aware distillation strategy that significantly reduces training cost with enhanced realism, outperforming DMD2 with only $2.1\%$ its training cost. Second, we introduce a joint training scheme where the model is fine-tuned with an RL objective while the timestep distillation training continues simultaneously. We demonstrate that the stable, well-defined loss from the ongoing distillation acts as a powerful regularizer, effectively stabilizing the RL training process and preventing policy collapse. Extensive experiments on score-based and flow matching models show that our proposed Flash-DMD not only converges significantly faster but also achieves state-of-the-art generation quality in the few-step sampling regime, outperforming existing methods in visual quality, human preference, and text-image alignment metrics. Our work presents an effective paradigm for training efficient, high-fidelity, and stable generative models. Codes are coming soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。