用扩散模型生成不想要的图像来反向设计负提示，提升图像生成质量。
Improving image synthesis with diffusion-negative sampling
- 通过扩散负采样生成与正提示最不匹配的图像。
- 新方法使负提示更符合模型语义，提升生成效果。
- 无需训练，兼容多种扩散模型，适合图像生成研究者。
在扩散模型（DMs）进行图像生成时，负提示n可补充文本提示p，帮助排除生成中不希望出现的属性。尽管这能提升提示遵循度和图像质量，但寻找有效负提示仍具挑战性。我们指出，这是由于人类与扩散模型间存在语义鸿沟，导致对模型有效的负提示对人类而言显得不直观。为此，我们提出新的扩散负提示策略（DNP），其核心是扩散负采样（DNS）——从模型分布中采样与正提示最不一致的图像。给定文本提示p，通过该过程生成一张不合规图像，再由用户或图文描述模型将其转化为自然语言负提示n*，最终以(p, n*)组合驱动扩散模型。DNS实现简单，无需训练。实验与人工评估表明，DNP在定量与定性指标上均表现优异，且可轻松集成至多个扩散模型变体。
原文摘要 · Abstract (English)
For image generation with diffusion models (DMs), a negative prompt n can be used to complement the text prompt p, helping define properties not desired in the synthesized image. While this improves prompt adherence and image quality, finding good negative prompts is challenging. We argue that this is due to a semantic gap between humans and DMs, which makes good negative prompts for DMs appear unintuitive to humans. To bridge this gap, we propose a new diffusion-negative prompting (DNP) strategy. DNP is based on a new procedure to sample images that are least compliant with p under the distribution of the DM, denoted as diffusion-negative sampling (DNS). Given p, one such image is sampled, which is then translated into natural language by the user or a captioning model, to produce the negative prompt n*. The pair (p, n*) is finally used to prompt the DM. DNS is straightforward to implement and requires no training. Experiments and human evaluations show that DNP performs well both quantitatively and qualitatively and can be easily combined with several DM variants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。