通过迭代优化提示词，让图像生成更贴合用户意图。
Test-time Prompt Refinement for Text-to-Image Models
- 用预训练多模态大模型分析生成图与提示词的偏差，动态调整提示词。
- 在多个数据集上显著提升图像与提示的对齐度和视觉一致性。
- 无需训练模型，可直接接入任意黑箱文本生成图像系统。
文本到图像（T2I）生成模型虽取得显著进展，但仍面临提示词敏感问题：提示词微小变化可能导致输出不一致或不准确。为解决这一挑战，我们提出一种无需额外训练底层T2I模型的闭环式测试时提示词优化框架，称为TIR。在该方法中，每轮生成后均进行一次优化步骤，由预训练的多模态大语言模型（MLLM）分析输出图像与用户提示。MLLM检测出错位（如缺失物体、属性错误），并生成更精确、物理合理的优化提示用于下一轮图像生成。通过反复迭代提示词优化与图文对齐验证，TIR能有效纠正错误，模拟人类艺术家的迭代创作过程。实验表明，该闭环策略在多个基准数据集上均提升了图文对齐度与视觉连贯性，同时保持与黑箱T2I模型的即插即用集成能力。
原文摘要 · Abstract (English)
Text-to-image (T2I) generation models have made significant strides but still struggle with prompt sensitivity: even minor changes in prompt wording can yield inconsistent or inaccurate outputs. To address this challenge, we introduce a closed-loop, test-time prompt refinement framework that requires no additional training of the underlying T2I model, termed TIR. In our approach, each generation step is followed by a refinement step, where a pretrained multimodal large language model (MLLM) analyzes the output image and the user's prompt. The MLLM detects misalignments (e.g., missing objects, incorrect attributes) and produces a refined and physically grounded prompt for the next round of image generation. By iteratively refining the prompt and verifying alignment between the prompt and the image, TIR corrects errors, mirroring the iterative refinement process of human artists. We demonstrate that this closed-loop strategy improves alignment and visual coherence across multiple benchmark datasets, all while maintaining plug-and-play integration with black-box T2I models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。