用思维链重写提示词,让AI图像生成更符合用户意图。
PromptEnhancer: A Simple Approach to Enhance Text-to-Image Models via Chain-of-Thought Prompt Rewriting
- 通过思维链机制重写提示词,不修改模型权重。
- 在HunyuanImage 2.1上显著提升图文对齐度,覆盖24类常见错误。
- 适合需要精准控制生成结果的研究者和开发者。
文本到图像(T2I)扩散模型虽能生成高质量图像,但在处理复杂提示时仍存在属性绑定、否定语义和组合关系理解偏差等问题,导致用户意图与生成结果严重不符。为此,我们提出PromptEnhancer,一种无需修改预训练模型权重的通用提示重写框架。该框架采用强化学习训练一个思维链(CoT)重写器,由专门设计的AlignEvaluator奖励模型提供细粒度反馈。AlignEvaluator基于对24种典型失败模式的系统分析构建,可评估提示的准确性。通过最大化该奖励,重写器学习生成更易被T2I模型正确解析的提示。在HunyuanImage 2.1上的实验表明,PromptEnhancer显著提升了多种语义与组合性挑战下的图像-文本对齐效果。此外,我们还引入了一个高质量的人工偏好基准,以推动该方向未来研究。
原文摘要 · Abstract (English)
Recent advancements in text-to-image (T2I) diffusion models have demonstrated remarkable capabilities in generating high-fidelity images. However, these models often struggle to faithfully render complex user prompts, particularly in aspects like attribute binding, negation, and compositional relationships. This leads to a significant mismatch between user intent and the generated output. To address this challenge, we introduce PromptEnhancer, a novel and universal prompt rewriting framework that enhances any pretrained T2I model without requiring modifications to its weights. Unlike prior methods that rely on model-specific fine-tuning or implicit reward signals like image-reward scores, our framework decouples the rewriter from the generator. We achieve this by training a Chain-of-Thought (CoT) rewriter through reinforcement learning, guided by a dedicated reward model we term the AlignEvaluator. The AlignEvaluator is trained to provide explicit and fine-grained feedback based on a systematic taxonomy of 24 key points, which are derived from a comprehensive analysis of common T2I failure modes. By optimizing the CoT rewriter to maximize the reward from our AlignEvaluator, our framework learns to generate prompts that are more precisely interpreted by T2I models. Extensive experiments on the HunyuanImage 2.1 model demonstrate that PromptEnhancer significantly improves image-text alignment across a wide range of semantic and compositional challenges. Furthermore, we introduce a new, high-quality human preference benchmark to facilitate future research in this direction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。