用单次优化提升文本图像生成对齐速度，实时性更强
Fast Prompt Alignment for Text-to-Image Generation
- 单次迭代通过大模型重写提示词，避免传统多轮调整
- 在COCO和PartiPrompts上实现接近峰值的对齐效果，耗时减少90%以上
- 适合需要快速生成的实时场景，如交互式设计工具
文本到图像生成技术发展迅速，但复杂提示与生成图像之间的对齐仍具挑战，尤其在对象关系和细节层面。本文提出快速提示对齐（FPA）框架，采用单次处理方式，在无需迭代的前提下提升对齐效率。FPA利用大语言模型（LLMs）进行单轮提示改写，随后通过微调或上下文学习使用优化后的提示，实现实时推理，显著降低计算开销同时保持对齐精度。在COCO Captions和PartiPrompts数据集上的大量评估表明，FPA以极小的处理时间开销达到与现有方法相当甚至更优的对齐性能，自动化指标（TIFA、VQA）和人工评估结果一致。专家标注的人工研究进一步验证了人工判断与自动评分高度相关，证明了FPA改进的可靠性。该方法为高时效性场景提供可扩展、高效的替代方案，代码已开源。
原文摘要 · Abstract (English)
Text-to-image generation has advanced rapidly, yet aligning complex textual prompts with generated visuals remains challenging, especially with intricate object relationships and fine-grained details. This paper introduces Fast Prompt Alignment (FPA), a prompt optimization framework that leverages a one-pass approach, enhancing text-to-image alignment efficiency without the iterative overhead typical of current methods like OPT2I. FPA uses large language models (LLMs) for single-iteration prompt paraphrasing, followed by fine-tuning or in-context learning with optimized prompts to enable real-time inference, reducing computational demands while preserving alignment fidelity. Extensive evaluations on the COCO Captions and PartiPrompts datasets demonstrate that FPA achieves competitive text-image alignment scores at a fraction of the processing time, as validated through both automated metrics (TIFA, VQA) and human evaluation. A human study with expert annotators further reveals a strong correlation between human alignment judgments and automated scores, underscoring the robustness of FPA's improvements. The proposed method showcases a scalable, efficient alternative to iterative prompt optimization, enabling broader applicability in real-time, high-demand settings. The codebase is provided to facilitate further research: https://github.com/tiktok/fast_prompt_alignment
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。