用新方法生成更优更多样提示,提升图像生成效果。
Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation
- 将提示优化转为概率采样问题,避免重复与单一
- 在多个奖励函数下生成高质量且多样的提示
- 适合需要可控图像生成的研究者和开发者
近期文本到图像扩散模型取得了显著进展,但如何通过用户意图等属性控制生成过程仍具挑战,这些属性可表达为黑箱奖励函数。本文聚焦提示适应,将原始提示优化为模型偏好提示以生成期望图像。以往工作采用强化学习优化提示,但常导致生成后缀相似、行为确定。为此,我们提出基于生成流网络(GFlowNets)的提示适应方法(PAG),将提示优化视为概率推断问题。核心思想是利用生成流网络从非归一化密度函数中采样,实现高质量与多样性并重的提示生成。然而,我们发现直接应用GFlowNets存在模式崩溃,并揭示了模型神经可塑性逐步丧失的现象,这在序列式提示生成中因信用分配效率低而加剧。为此,我们在PAG中系统性地引入流动再激活、奖励优先采样与奖励分解机制。大量实验表明,PAG能有效学习生成优质且多样的提示。同时,PAG在多种奖励函数下表现稳健,并具备跨不同文本到图像模型的迁移能力。
原文摘要 · Abstract (English)
Recent advances in text-to-image diffusion models have achieved impressive image generation capabilities. However, it remains challenging to control the generation process with desired properties (e.g., aesthetic quality, user intention), which can be expressed as black-box reward functions. In this paper, we focus on prompt adaptation, which refines the original prompt into model-preferred prompts to generate desired images. While prior work uses reinforcement learning (RL) to optimize prompts, we observe that applying RL often results in generating similar postfixes and deterministic behaviors. To this end, we introduce \textbf{P}rompt \textbf{A}daptation with \textbf{G}FlowNets (\textbf{PAG}), a novel approach that frames prompt adaptation as a probabilistic inference problem. Our key insight is that leveraging Generative Flow Networks (GFlowNets) allows us to shift from reward maximization to sampling from an unnormalized density function, enabling both high-quality and diverse prompt generation. However, we identify that a naive application of GFlowNets suffers from mode collapse and uncovers a previously overlooked phenomenon: the progressive loss of neural plasticity in the model, which is compounded by inefficient credit assignment in sequential prompt generation. To address this critical challenge, we develop a systematic approach in PAG with flow reactivation, reward-prioritized sampling, and reward decomposition for prompt adaptation. Extensive experiments validate that PAG successfully learns to sample effective and diverse prompts for text-to-image generation. We also show that PAG exhibits strong robustness across various reward functions and transferability to different text-to-image models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。