动态调整生成引导力度,让AI画画更贴合提示词需求。
Dynamic Classifier-Free Diffusion Guidance via Online Feedback
- 根据每步生成质量实时反馈,动态选择最佳引导系数。
- 在Imagen 3上实现53.8%人类偏好胜率,文本生成提升至55.5%。
- 无需复杂调参,适配各类提示词,通用性强。
Classifier-free guidance(CFG)是文生图扩散模型的核心技术,但其效果受限于静态引导尺度。当前‘一刀切’策略无法适应不同提示词的需求;而基于梯度的修正或固定启发式调度方案则引入额外复杂性且泛化能力差。本文提出一种动态CFG调度框架,利用一系列小型潜空间评估器(如CLIP对齐度、判别器保真度、人类偏好奖励模型)在反向扩散过程每一步提供在线反馈,通过贪婪搜索为每个时间步选取最优引导尺度,生成针对特定提示词和样本的个性化引导序列。实验在小模型及前沿模型Imagen 3上验证了该方法的有效性,显著提升文本对齐、视觉质量、文本渲染与数值推理能力。相比Imagen 3默认基线,本方法在整体偏好上达到53.8%的人类偏好胜率,针对文本渲染等特定任务更达55.5%。结果表明最优引导策略本质应为动态且提示词依赖,本文提供了高效通用的实现框架。
原文摘要 · Abstract (English)
Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This "one-size-fits-all" approach fails to adapt to the diverse requirements of different prompts; moreover, prior solutions like gradient-based correction or fixed heuristic schedules introduce additional complexities and fail to generalize. In this work, we challeng this static paradigm by introducing a framework for dynamic CFG scheduling. Our method leverages online feedback from a suite of general-purpose and specialized small-scale latent-space evaluations, such as CLIP for alignment, a discriminator for fidelity and a human preference reward model, to assess generation quality at each step of the reverse diffusion process. Based on this feedback, we perform a greedy search to select the optimal CFG scale for each timestep, creating a unique guidance schedule tailored to every prompt and sample. We demonstrate the effectiveness of our approach on both small-scale models and the state-of-the-art Imagen 3, showing significant improvements in text alignment, visual quality, text rendering and numerical reasoning. Notably, when compared against the default Imagen 3 baseline, our method achieves up to 53.8% human preference win-rate for overall preference, a figure that increases up to to 55.5% on prompts targeting specific capabilities like text rendering. Our work establishes that the optimal guidance schedule is inherently dynamic and prompt-dependent, and provides an efficient and generalizable framework to achieve it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。