分析GPT-4生成的教师评价，发现其存在性别刻板印象。
Unveiling Gender Bias in Large Language Models: Using Teacher's Evaluation in Higher Education As an Example
- 用多种方法分析六学科教师评价文本中的性别语言差异
- 女性被更多关联亲和、支持类词汇，男性更常被关联娱乐类词汇
- 适合关注AI公平性、教育评价与社会偏见的研究者阅读
本文研究大语言模型（LLM）在高等教育情境下生成的教师评价中存在的性别偏见，聚焦GPT-4在六个学科中生成的评价内容。通过综合运用几率比（OR）分析、词嵌入关联测试（WEAT）、情感分析与上下文分析框架，识别出反映社会刻板印象的性别相关语言模式。具体而言，描述亲和力与支持性的词汇更常用于女性教师，而描述娱乐性的词汇则主要指向男性教师，这与共情性（communal）和主导性（agentic）行为概念一致。研究还发现，男性特征形容词与男性姓名间存在中度至强相关性，但职业与家庭相关词汇未明显体现性别偏见。这些结果与既有社会规范研究一致，表明大语言模型生成文本会折射现有社会偏见。
原文摘要 · Abstract (English)
This paper investigates gender bias in Large Language Model (LLM)-generated teacher evaluations in higher education setting, focusing on evaluations produced by GPT-4 across six academic subjects. By applying a comprehensive analytical framework that includes Odds Ratio (OR) analysis, Word Embedding Association Test (WEAT), sentiment analysis, and contextual analysis, this paper identified patterns of gender-associated language reflecting societal stereotypes. Specifically, words related to approachability and support were used more frequently for female instructors, while words related to entertainment were predominantly used for male instructors, aligning with the concepts of communal and agentic behaviors. The study also found moderate to strong associations between male salient adjectives and male names, though career and family words did not distinctly capture gender biases. These findings align with prior research on societal norms and stereotypes, reinforcing the notion that LLM-generated text reflects existing biases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。