提出细粒度图文评估方法，精准检测生成图像与提示的语义一致性。
T2I-FineEval: Fine-Grained Compositional Metric for Text-to-Image Evaluation
- 将提示分解为细粒度问题，逐项评估图像内容
- 在多个数据集上超越现有最佳指标表现
- 适合研究文本到图像生成模型的学者使用
尽管近期文本到图像生成模型已取得显著进展，但仍难以准确捕捉提示中的属性绑定和实体间空间关系等组合复杂性。现有评估指标如CLIPScore无法揭示此类偏差。已有工作通过视觉问答（VQA）分解提示以提升评估鲁棒性，但仍未能充分覆盖图像内的组合性。为此，本文提出一种新评估指标，将图像拆解为组件，将文本转化为关于生成图像的细粒度问题进行评估。实验表明，该方法优于当前最优指标，有效提升了对文本到图像生成模型的评估能力。代码已公开于 https://github.com/hadi-hosseini/T2I-FineEval。
原文摘要 · Abstract (English)
Although recent text-to-image generative models have achieved impressive performance, they still often struggle with capturing the compositional complexities of prompts including attribute binding, and spatial relationships between different entities. This misalignment is not revealed by common evaluation metrics such as CLIPScore. Recent works have proposed evaluation metrics that utilize Visual Question Answering (VQA) by decomposing prompts into questions about the generated image for more robust compositional evaluation. Although these methods align better with human evaluations, they still fail to fully cover the compositionality within the image. To address this, we propose a novel metric that breaks down images into components, and texts into fine-grained questions about the generated image for evaluation. Our method outperforms previous state-of-the-art metrics, demonstrating its effectiveness in evaluating text-to-image generative models. Code is available at https://github.com/hadi-hosseini/ T2I-FineEval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。