通过细粒度优化提升文本图像生成中概念的忠实度。
No Concept Left Behind: Test-Time Optimization for Compositional Text-to-Image Generation
- 将提示分解为语义概念，分层评估图文对齐
- 在DrawBench和CompBench上概念覆盖率提升23.6%
- 适合需要高精度图像生成的科研与设计场景
尽管文本到图像（T2I）模型取得进展，复杂提示中的元素仍常被遗漏或错误呈现。测试时优化作为一种无需重训练即可改进生成质量的方法受到关注。本文提出一种细粒度测试时优化框架，提升T2I生成的组合忠实度。不同于以往仅依赖全局图像/文本相似度的方法，我们把输入提示分解为语义概念，在全局与概念层面同时评估对齐情况。使用细粒度版CLIP计算概念级对应关系，生成关于缺失或错误概念的详细反馈，并将其输入迭代提示优化循环，由大语言模型生成改进后的提示。在DrawBench和CompBench上的实验表明，该方法在概念覆盖率和人工评价的忠实度上显著优于标准测试时优化及基础T2I模型。代码已开源。
原文摘要 · Abstract (English)
Despite recent advances in text-to-image (T2I) models, they often fail to faithfully render all elements of complex prompts, frequently omitting or misrepresenting specific objects and attributes. Test-time optimization has emerged as a promising approach to address this limitation by refining generation without the need for retraining. In this paper, we propose a fine-grained test-time optimization framework that enhances compositional faithfulness in T2I generation. Unlike most of prior approaches that rely solely on a global image/text similarity score, our method decomposes the input prompt into semantic concepts and evaluates alignment at both the global and concept levels. A fine-grained variant of CLIP is used to compute concept-level correspondence, producing detailed feedback on missing or inaccurate concepts. This feedback is fed into an iterative prompt refinement loop, enabling the large language model to propose improved prompts. Experiments on DrawBench and CompBench prompts demonstrate that our method significantly improves concept coverage and human-judged faithfulness over both standard test-time optimization and the base T2I model. Code is available at: https://github.com/AmirMansurian/NoConceptLeftBehind
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。