测试十国文化图像生成能力，发现主流模型存在显著文化偏见。
Diffusion Models Through a Global Lens: Are They Culturally Inclusive?
- 构建跨文化图像评测基准CultDiff，覆盖十国文化元素。
- 模型在建筑、服饰、食物等文化细节上生成效果差，尤其弱势地区更明显。
- 提出CultDiff-S模型，可预测人类对文化图像的评价偏好。
文本到图像的扩散模型虽能生成视觉逼真的图像，但其对文化细节的准确表达仍存疑问。本文提出CultDiff基准，评估先进扩散模型在十个不同国家文化图像生成上的表现。通过细粒度分析相似性维度，发现模型在文化相关性、描述准确性与真实感方面与真实参考图像存在显著差距，尤其在代表性不足的国家区域更为明显。结合人工评价数据，我们开发了基于神经网络的图像-图像相似性度量CultDiff-S，能有效预测人类对含文化元素图像的判断。研究强调需构建更具包容性的生成式AI系统和更均衡的文化数据集。
原文摘要 · Abstract (English)
Text-to-image diffusion models have recently enabled the creation of visually compelling, detailed images from textual prompts. However, their ability to accurately represent various cultural nuances remains an open question. In our work, we introduce CultDiff benchmark, evaluating state-of-the-art diffusion models whether they can generate culturally specific images spanning ten countries. We show that these models often fail to generate cultural artifacts in architecture, clothing, and food, especially for underrepresented country regions, by conducting a fine-grained analysis of different similarity aspects, revealing significant disparities in cultural relevance, description fidelity, and realism compared to real-world reference images. With the collected human evaluations, we develop a neural-based image-image similarity metric, namely, CultDiff-S, to predict human judgment on real and generated images with cultural artifacts. Our work highlights the need for more inclusive generative AI systems and equitable dataset representation over a wide range of cultures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。