通过弱模型与强模型的差异反射,提升生成质量与真实数据一致性。
Weak-to-Strong Diffusion with Reflection
- 利用弱-强模型差异进行反射操作,引导采样走向真实数据分布。
- 在图像、视频等多模态上实现SOTA,HPSv2胜率提升至90%。
- 灵活适配不同模型对,计算开销小,适合实际部署。
扩散生成模型的目标是通过梯度得分匹配使学习到的分布与真实数据分布对齐。然而,训练数据质量、建模策略和架构设计的固有限制,导致生成结果与真实数据之间存在不可避免的差距。为此,本文提出弱到强扩散(W2SD)框架,利用现有弱模型与强模型之间的估计差异(即弱-强差异)来弥合理想模型与强模型之间的鸿沟。通过交替执行去噪与反演操作,并结合弱-强差异的反射机制,理论上可使潜在变量沿采样轨迹向真实数据分布区域移动。W2SD具有高度灵活性和广泛适用性,可通过选择不同弱-强模型对实现多样改进(如DreamShaper vs. SD1.5,MoE中优秀专家与劣质专家)。大量实验表明,W2SD显著提升人类偏好、美学质量和提示遵循度,在多种模态(如图像、视频)、架构(如UNet-based、DiT-based、MoE)和基准上均达到最先进水平。例如,使用W2SD的Juggernaut-XL在HPSv2评测中胜率最高可达原结果的90%。此外,性能提升远超额外计算开销,不同弱-强差异带来的累积增益进一步证明其实用性和可部署性。
原文摘要 · Abstract (English)
The goal of diffusion generative models is to align the learned distribution with the real data distribution through gradient score matching. However, inherent limitations in training data quality, modeling strategies, and architectural design lead to inevitable gap between generated outputs and real data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel framework that utilizes the estimated difference between existing weak and strong models (i.e., weak-to-strong difference) to bridge the gap between an ideal model and a strong model. By employing a reflective operation that alternates between denoising and inversion with weak-to-strong difference, we theoretically understand that W2SD steers latent variables along sampling trajectories toward regions of the real data distribution. W2SD is highly flexible and broadly applicable, enabling diverse improvements through the strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5, good experts vs. bad experts in MoE). Extensive experiments demonstrate that W2SD significantly improves human preference, aesthetic quality, and prompt adherence, achieving SOTA performance across various modalities (e.g., image, video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to 90% over the original results. Moreover, the performance gains achieved by W2SD markedly outweigh its additional computational overhead, while the cumulative improvements from different weak-to-strong difference further solidify its practical utility and deployability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。