评测文生图模型在跨文化日常活动中的表现，发现南北差异与偏见问题。
Culture in Action: Evaluating Text-to-Image Models through Social Activities
- 构建跨文化活动基准CULTIVate，涵盖16国576个场景
- 发现模型对全球北方国家表现更好，南方国家易出现幻觉
- 提出可解释评价框架，适合评估文化公平性
文生图扩散模型通过大规模网络数据训练获得逼真效果，但继承文化偏见，难以真实呈现弱势地区文化。现有文化评测多聚焦物品类（如食物、服饰、建筑），忽略更反映文化规范的社交与日常活动。缺乏有效度量指标。本文提出CULTIVate基准，评估文生图模型在跨文化活动（如问候、用餐、游戏、传统舞蹈、节庆）上的表现，覆盖16个国家，包含576个提示与超过19,000张图像，提供基于描述符的可解释评价框架，涵盖背景、服饰、物品与互动等文化维度。提出四项指标：文化一致性、幻觉程度、夸张元素与多样性。结果揭示系统性差异：模型在发达国家表现优于发展中国家，不同模型失败模式各异。人类评估验证，本方法指标与人工判断相关性高于现有文本-图像度量。
原文摘要 · Abstract (English)
Text-to-image (T2I) diffusion models achieve impressive photorealism by training on large-scale web data, but models inherit cultural biases and fail to depict underrepresented regions faithfully. Existing cultural benchmarks focus mainly on object-centric categories (e.g., food, attire, and architecture), overlooking the social and daily activities that more clearly reflect cultural norms. Few metrics exist for measuring cultural faithfulness. We introduce CULTIVate, a benchmark for evaluating T2I models on cross-cultural activities (e.g., greetings, dining, games, traditional dances, and cultural celebrations). CULTIVate spans 16 countries with 576 prompts and more than 19,000 images, and provides an explainable descriptor-based evaluation framework across multiple cultural dimensions, including background, attire, objects, and interactions. We propose four metrics to measure cultural alignment, hallucination, exaggerated elements, and diversity. Our findings reveal systematic disparities: models perform better for global north countries than for the global south, with distinct failure modes across T2I systems. Human studies confirm that our metrics correlate more strongly with human judgments than existing text-image metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。