将分布匹配重定义为奖励,提升扩散模型蒸馏的效率与稳定性。
$R_\text{dm}$: Re-conceptualizing Distribution Matching as a Reward for Diffusion Distillation
- 把分布匹配设计为奖励信号,统一蒸馏与强化学习框架。
- 引入分组归一化增强优化稳定,FID降低1.87,采样效率显著提升。
- 适合追求高效高保真生成的科研与工业应用,尤其关注实时生成场景。
扩散模型生成性能顶尖,但受限于缓慢的迭代采样过程。扩散蒸馏技术虽能实现高保真少步生成,但传统目标常因过度依赖教师模型而限制学生表现。近期方法尝试结合强化学习(RL),通常采用简单叠加蒸馏与RL目标。本文提出新范式:将分布匹配重新定义为奖励 $R_ ext{dm}$,打通扩散匹配蒸馏(DMD)与强化学习间的算法鸿沟,带来三大优势:(1) 优化稳定性增强:提出分组归一化分布匹配(GNDM),利用组均值统计稳定 $R_ ext{dm}$ 估计,提供更可靠的优化方向;(2) 奖励融合无缝:奖励中心框架天然支持自适应加权,可灵活结合外部奖励模型;(3) 采样效率提升:遵循强化学习原则,可集成重要性采样(IS),显著提升采样效率。大量实验表明,GNDM优于原始 DMD,FID降低1.87;多奖励变体 GNDMR 在美学质量与保真度间取得最佳平衡,达成峰值 HPS 30.37 与低 FID-SD 12.21。最终,$R_ ext{dm}$ 构建了一个灵活、稳定且高效的实时高保真合成框架。代码即将发布。
原文摘要 · Abstract (English)
Diffusion models achieve state-of-the-art generative performance but are fundamentally bottlenecked by their slow, iterative sampling process. While diffusion distillation techniques enable high-fidelity, few-step generation, traditional objectives often restrict the student's performance by anchoring it solely to the teacher. Recent approaches have attempted to break this ceiling by integrating Reinforcement Learning (RL), typically through a simple summation of distillation and RL objectives. In this work, we propose a novel paradigm by re-conceptualizing distribution matching as a reward, denoted as $R_\text{dm}$. This unified perspective bridges the algorithmic gap between Diffusion Matching Distillation (DMD) and RL, providing several primary benefits. (1) Enhanced Optimization Stability: We introduce Group Normalized Distribution Matching (GNDM), which adapts standard RL group normalization to stabilize $R_\text{dm}$ estimation. By leveraging group-mean statistics, GNDM establishes a more robust and effective optimization direction. (2) Seamless Reward Integration: Our reward-centric formulation inherently supports adaptive weighting mechanisms, allowing for the fluid combination of DMD with external reward models. (3) Improved Sampling Efficiency: By aligning with RL principles, the framework readily incorporates Importance Sampling (IS), leading to a significant boost in sampling efficiency. Extensive experiments demonstrate that GNDM outperforms vanilla DMD, reducing the FID by 1.87. Furthermore, our multi-reward variant, GNDMR, surpasses existing baselines by striking an optimal balance between aesthetic quality and fidelity, achieving a peak HPS of 30.37 and a low FID-SD of 12.21. Ultimately, $R_\text{dm}$ provides a flexible, stable, and efficient framework for real-time, high-fidelity synthesis. Codes are coming soon.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。