人类能识别出AI图像生成中的复杂组合缺陷，研究构建了首个相关数据集。
When Composition Doesn't Add Up: Humans Identifying Defects in AI-Generated Images
- 通过人工编辑提示词构造复杂组合场景，构建测试用图像数据集。
- 29人对651张图进行多标签标注，发现模型在多重实体与属性组合时缺陷显著。
- 数据集可训练模型预测并优化生成缺陷，适合评估与改进AI图像质量。
当前最先进的文本到图像（T2I）模型在涉及多重实体和属性的复杂组合提示下，表现出明显且系统性的缺陷。本文研究人类如何识别此类缺陷：从人物、手部、物体和场景四类中手动选取651张具有复杂组合特征的参考图像，通过编辑ChatGPT生成的提示词构建组合性提示；将提示输入三种选定的T2I模型生成图像，并开展全面主观评估，每位图像由29名参与者提供多标签缺陷类型与位置标注。研究构建了组合性AI生成图像缺陷数据集（CO-AID），包含参考图像、提示、生成图像及缺陷标注信息。实验表明，在CO-AID上训练的深度模型不仅能有效预测生成缺陷，还可用于优化图像生成过程，验证了其可用性与有效性。数据集及补充材料已公开于https://github.com/Future-IQA/CO-AID。
原文摘要 · Abstract (English)
*Chulin Zhao and Ruoqi Hu contributed equally to this work. State-of-the-art text-to-image (T2I) models exhibit pronounced and systematic defects when prompts involve intricate compositional factors such as multiple entities and multiple attributes. In this paper, we investigate how humans identify such defects. Specifically, we manually select 651 reference images from the four categories of people, hand, object, and scene that exhibit complex compositional characteristics, from which prompts emphasizing compositional factors are derived by manually editing ChatGPT-generated prompts. We then feed the prompts into three selected T2I models to generate AI images and conduct a comprehensive subjective study to identify their defects. For each image, 29 participants provide multi-label assessments specifying defect types and locations. The study yields the compositional AI-generated image defect (CO-AID) dataset, including reference images, prompts, AI-generated images, and information on defect locations and types. Experimental results show that training a deep model on CO-AID can both predict defects in AI-generated images and optimize AI image generation, demonstrating its usability and effectiveness. The database and supplementary materials are available at: https://github.com/Future-IQA/CO-AID .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。