arXiv:2412.04300cs.CVcs.AI2024-12ACL被引 14

首个针对知识密集型概念生成的事实性评估基准,检验AI图像是否符合真实知识。

T2I-FactualBench: Benchmarking the Factuality of Text-to-Image Models with Knowledge-Intensive Concepts

  • 构建三层次知识密集型图文生成框架,涵盖单概念记忆到多概念组合
  • 设计多轮视觉问答评估体系,发现当前顶尖模型事实性仍有巨大提升空间
  • 适用于研究模型知识理解能力的学者,尤其关注生成内容真实性的人

文本到图像(T2I)生成的质量评估仍是重大挑战。现有研究多聚焦于图文对齐、图像质量与物体构图,较少关注知识密集型概念生成的事实性。为此,本文提出T2I-FactualBench——目前规模最大的知识密集型概念生成事实性评估基准,包含大量概念与针对性提示。该基准采用三层知识密集型图文生成框架,从单一概念记忆到多概念复合;并引入多轮视觉问答(VQA)评估体系,量化评估三层次任务中的事实性表现。实验表明,当前最先进(SOTA)的T2I模型在事实性方面仍存在显著改进空间。

原文摘要 · Abstract (English)

Evaluating the quality of synthesized images remains a significant challenge in the development of text-to-image (T2I) generation. Most existing studies in this area primarily focus on evaluating text-image alignment, image quality, and object composition capabilities, with comparatively fewer studies addressing the evaluation of the factuality of T2I models, particularly when the concepts involved are knowledge-intensive. To mitigate this gap, we present T2I-FactualBench in this work - the largest benchmark to date in terms of the number of concepts and prompts specifically designed to evaluate the factuality of knowledge-intensive concept generation. T2I-FactualBench consists of a three-tiered knowledge-intensive text-to-image generation framework, ranging from the basic memorization of individual knowledge concepts to the more complex composition of multiple knowledge concepts. We further introduce a multi-round visual question answering (VQA) based evaluation framework to assess the factuality of three-tiered knowledge-intensive text-to-image generation tasks. Experiments on T2I-FactualBench indicate that current state-of-the-art (SOTA) T2I models still leave significant room for improvement.

图文生成事实性评估知识密集型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。