无需训练即可提升扩散与流模型生成质量,自动抑制低质图像。
Self-Guidance: Boosting Flow and Diffusion Generation on Their Own
- 利用不同噪声水平下的采样得分函数自引导优化生成质量。
- 在多个指标上超越现有方法,手、脸等人体结构更真实。
- 可即插即用,效率提升50%,适合各类扩散与流模型。
高质量生成依赖于恰当的引导策略,而现有方法通常需要特定训练或强先验假设,限制了适用性。本文提出自引导(Self-Guidance, SG),基于从高噪声到低噪声阶段密度显著下降的现象,检测并抑制低质量样本生成。SG仅使用原始扩散或流模型在不同噪声水平下的采样得分函数,无需额外训练,具备高度灵活性,可直接嵌入任意扩散或流模型中。我们进一步提出高效变体SG-prev,复用前一步输出,避免额外前向传播,效率提升50%。在多种架构(包括UNet与Transformer)及文本到图像、文本到视频任务上进行大量实验,基于Stable Diffusion 3.5和FLUX等开源模型,SG在FID和人类偏好评分等多项指标上均优于现有方法。尤其对双手、面部、手臂等人身体结构生成效果显著改善,有效消除常见伪影。代码已开源。
原文摘要 · Abstract (English)
Proper guidance strategies are essential to achieve high-quality generation results without retraining diffusion and flow-based text-to-image models. Existing guidance either requires specific training or strong inductive biases of diffusion model networks, which potentially limits their ability and application scope. Motivated by the observation that artifact outliers can be detected by a significant decline in the density from a noisier to a cleaner noise level, we propose Self-Guidance (SG), which can significantly improve the quality of the generated image by suppressing the generation of low-quality samples. The biggest difference from existing guidance is that SG only relies on the sampling score function of the original diffusion or flow model at different noise levels, with no need for any tricky and expensive guidance-specific training. This makes SG highly flexible to be used in a plug-and-play manner by any diffusion or flow models. We also introduce an efficient variant of SG, named SG-prev, which reuses the output from the immediately previous diffusion step to avoid additional forward passes of the diffusion network.We conduct extensive experiments on text-to-image and text-to-video generation with different architectures, including UNet and transformer models. With open-sourced diffusion models such as Stable Diffusion 3.5 and FLUX, SG exceeds existing algorithms on multiple metrics, including both FID and Human Preference Score. SG-prev also achieves strong results over both the baseline and the SG, with 50 percent more efficiency. Moreover, we find that SG and SG-prev both have a surprisingly positive effect on the generation of physiologically correct human body structures such as hands, faces, and arms, showing their ability to eliminate human body artifacts with minimal efforts. We have released our code at https://github.com/maple-research-lab/Self-Guidance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。