用奥运男女赛事数据揭示大模型在性别模糊时对女性的系统性偏见。
Sports and Women's Sports: Gender Bias in Text Generation with Olympic Data
- 通过对比奥运会男女赛事数据，检测模型生成文本中的性别偏见。
- 当提示中性别不明确时，模型90%以上仅输出男性赛事结果。
- 适合关注AI伦理、公平性研究的研究者和开发者参考。
大型语言模型（LLMs）在以往研究中已被证实存在偏见，其生成内容往往符合世界刻板印象，或未能反映历史上被边缘化群体的观点与价值。本文利用奥运会男女赛事的并行数据，探究语言模型中的多种性别偏见。我们定义了三项度量指标，发现当提示中性别模糊时，模型普遍偏向男性：频繁仅检索并输出男性赛事结果，甚至未明确标注其为男性赛事。这揭示了在体育语境下，大模型存在广泛而深刻的性别偏见。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have been shown to be biased in prior work, as they generate text that is in line with stereotypical views of the world or that is not representative of the viewpoints and values of historically marginalized demographic groups. In this work, we propose using data from parallel men's and women's events at the Olympic Games to investigate different forms of gender bias in language models. We define three metrics to measure bias, and find that models are consistently biased against women when the gender is ambiguous in the prompt. In this case, the model frequently retrieves only the results of the men's event with or without acknowledging them as such, revealing pervasive gender bias in LLMs in the context of athletics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。