构建首个长文本生成图像的可解释评估基准,支持细粒度对齐分析。
LongT2IBench: A Benchmark for Evaluating Long Text-to-Image Generation with Graph-structured Annotations
- 设计生成-精炼-评分流程,将长文本转为实体关系图结构
- 提供14000组图文对与图结构标注,支持细粒度对齐分析
- 提出LongT2IExpert模型,实现评分与结构化解释双重输出
长文本生成图像(Long T2I)的兴起迫切需要自动且可解释的评估模型来衡量图像与长提示间的对齐程度。然而现有评测基准多聚焦短提示场景,仅提供平均评分或李克特量表,难以支持长文本场景下的可解释性评估。本文提出LongT2IBench,包含14,000组长文本-图像对及图结构化人类标注。针对长提示的细节密集特性,我们设计了「生成-精炼-评分」标注协议,将其转化为包含实体、属性和关系的文本图结构,实现基于细粒度元素的对齐标注。最终,图结构标注被转换为对齐分数与解释信息,用于支持评估模型设计。基于此基准,我们进一步提出LongT2IExpert,一种利用层次化对齐思维链(CoT)进行指令微调的多模态大语言模型,使MLLM能同时输出量化评分与结构化解释。大量实验表明,LongT2IExpert在对齐评估与解释能力上均表现更优。数据与代码已公开于 https://welldky.github.io/LongT2IBench-Homepage/。
原文摘要 · Abstract (English)
The increasing popularity of long Text-to-Image (T2I) generation has created an urgent need for automatic and interpretable models that can evaluate the image-text alignment in long prompt scenarios. However, the existing T2I alignment benchmarks predominantly focus on short prompt scenarios and only provide MOS or Likert scale annotations. This inherent limitation hinders the development of long T2I evaluators, particularly in terms of the interpretability of alignment. In this study, we contribute LongT2IBench, which comprises 14K long text-image pairs accompanied by graph-structured human annotations. Given the detail-intensive nature of long prompts, we first design a Generate-Refine-Qualify annotation protocol to convert them into textual graph structures that encompass entities, attributes, and relations. Through this transformation, fine-grained alignment annotations are achieved based on these granular elements. Finally, the graph-structed annotations are converted into alignment scores and interpretations to facilitate the design of T2I evaluation models. Based on LongT2IBench, we further propose LongT2IExpert, a LongT2I evaluator that enables multi-modal large language models (MLLMs) to provide both quantitative scores and structured interpretations through an instruction-tuning process with Hierarchical Alignment Chain-of-Thought (CoT). Extensive experiments and comparisons demonstrate the superiority of the proposed LongT2IExpert in alignment evaluation and interpretation. Data and code have been released in https://welldky.github.io/LongT2IBench-Homepage/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。