arXiv:2608.24112cs.AI2026-08

为文本生成图像模型提供细粒度评估,精准定位失败原因并智能选型。

Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing

论文配图:Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing
图 1 · 摘自论文原文
  • 将提示分解为带依赖关系的原子问题,用场景图结构建模
  • 联合完成率从80.7%降至37.2%,揭示复杂需求下的性能衰减
  • 无需训练即可实现成本敏感路由,节省21.3%算力

当前文本到图像模型总分相近但优势各异,难以实用选型。现有细粒度基准常将问题返回为提示分数或固定类别,削弱归因且忽略复杂性。本文提出QC-T2I-Bench,将开放提示转换为可归因的原子问题,并用戴维森场景图(DSG)组织其依赖关系。通过层次约束的问题聚合,前置失败则剔除下游问题,避免简单与复杂提示权重相同。利用DSG结构衡量提示内联合成功并比对重复实体,区分基础实现失败与附加要求下的失败。在中英文提示上评估多个开源模型,结果支持可靠排序与细粒度诊断:具备两项能力的组件联合完成率达80.7%,七项及以上时降至37.2%。最后复用相同记录实现无训练路由;我们的成本感知路由器以21.3%更少的GPU-s/MP达到ERNIE 89.51分的估计表现。

原文摘要 · Abstract (English)

Modern text-to-image (T2I) models often have similar total scores but different strengths, making practical selection difficult. Fine-grained benchmarks decompose prompts into questions, yet often return them to prompt scores and fixed categories, weakening attribution and ignoring complexity. Related requirements are also scored separately or as one total, obscuring basic versus compositional failure. We present QC-T2I-Bench, a question-centric framework that converts open prompts into attributed atomic questions and organizes their dependencies with Davidsonian Scene Graphs (DSGs). We use hierarchy-constrained question aggregation to exclude downstream questions after a prerequisite fails and to prevent simple and complex prompts from receiving the same total weight. We then use the DSG structure to measure joint success within prompts and compare repeated entities across prompts, separating basic realization failures from failures under additional requirements. We evaluate multiple open-source T2I models on English and Chinese prompts. The resulting question-level evidence supports reliable ranking and fine-grained diagnosis: joint completion falls from 80.7\% for components with two capabilities to 37.2\% for those with seven or more. Finally, we reuse the same records for training-free routing; our cost-aware router matches ERNIE's 89.51-point estimate with 21.3\% less GPU-s/MP.

图像生成评估框架细粒度诊断成本优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。