LLMs生成本地语言媒体时男性偏见更严重，需本地化测试
LLMs Mirror Country-Specific Gender Patterns If Asked, but Skew Male When Generating Media in Local Languages
- 在本地语言生成中，模型性别倾向偏离调查数据，男性化加剧
- 非美国地区模型在英语提示下偏见变化不显著，易被忽略
- 需本地化基准+生成式测试，才能发现真实性别偏见
大型语言模型（LLMs）越来越多用于媒体生成，但其是否延续性别刻板印象尚不清楚：现有基准多基于选择题格式，缺乏长文本生成评估，且非西方地区的本地性别关联基线数据稀缺。本研究从美国、印度、肯尼亚和尼日利亚共695名受访者中收集了22种职业与家庭角色的性别关联数据，并评估了八种LLMs在直接提问与媒体生成两种情境下的表现。模型在直接提问时能准确反映调查结果，但在本地语言媒体生成中显著偏向男性，与人类创作媒体中的男性偏见一致。在非美国地区，英语提示下的偏移较小且不显著，因此仅用英语或全球通用评估会漏掉模型在主要部署语言中的偏见。指令提示可部分缓解偏移方向，但牺牲了与调查数据的对齐度。因此，评估全球化部署中LLM的性别偏见，必须包含生成式测试、本地语言提示和本地人类基准。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to generate media, but whether their content perpetuates gender stereotypes is unknown: standard benchmarks rely on selection-based formats rather than long-form generation, and surveyed baselines for local gender associations are scarce outside the West. We collect gender associations for 22 occupational and domestic roles from 695 respondents across the United States, India, Kenya, and Nigeria, and evaluate eight LLMs under two regimes: direct questioning and media generation. Models track the surveyed associations under direct questioning but skew substantially more male under media generation in major local-language cells, consistent with the male bias documented in human-produced media. Outside the US, the shift is much smaller and non-significant under English prompting, so English-only or country-agnostic evaluation would miss this bias in the languages where these models are most deployed. Instruction prompting reduces the shift directionally, but trades off against alignment with the surveyed associations. Evaluating LLM gender bias for global deployment therefore requires generation-format testing, local-language prompting, and locally-collected human baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。