通过逐步细化提示词，让图像生成更贴合复杂文本描述。
Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Models
- 将长提示分解为从粗到细的多个子提示，逐步引入细节。
- 在GenAI-Bench数据集上，83%的提示下VQA得分提升超8分。
- 无需训练，可直接插入现有模型，适合需要精准对齐的场景。
文本到图像生成模型在处理包含复杂场景、多样物体及其视觉特征与空间关系的长提示时表现不佳。本文提出SCoPE（粗到细提示嵌入的调度插值）方法，一种无需训练的改进技术，通过逐步细化输入提示来提升文本与图像的对齐效果。给定详细提示后，先将其分解为多个子提示，这些子提示从描述整体场景布局逐渐过渡到精细细节。推理时，通过插值不同子提示，逐步将更细致的信息融入生成图像中。该方法为即插即用的无训练方案，在GenAI-Bench数据集上，83%的提示下，相比Stable Diffusion基线，平均VQA得分提升超过8分。
原文摘要 · Abstract (English)
Text-to-image generative models often struggle with long prompts detailing complex scenes, diverse objects with distinct visual characteristics and spatial relationships. In this work, we propose SCoPE (Scheduled interpolation of Coarse-to-fine Prompt Embeddings), a training-free method to improve text-to-image alignment by progressively refining the input prompt in a coarse-to-fine-grained manner. Given a detailed input prompt, we first decompose it into multiple sub-prompts which evolve from describing broad scene layout to highly intricate details. During inference, we interpolate between these sub-prompts and thus progressively introduce finer-grained details into the generated image. Our training-free plug-and-play approach significantly enhances prompt alignment, achieves an average improvement of more than +8 in Visual Question Answering (VQA) scores over the Stable Diffusion baselines on 83% of the prompts from the GenAI-Bench dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。