对比四款大模型在招聘面试报告中的隐性偏见，发现匿名化可减偏但效果因模型而异。
Revealing Hidden Bias in AI: Lessons from Large Language Models
- 通过对比去标识化与原始报告，检测模型在性别、种族、年龄上的偏见
- 匿名化显著降低性别偏见，但对种族和年龄偏见效果有限，模型间差异明显
- Llama 3.1 405B 偏见最低，为选型提供实证依据
随着大型语言模型（LLMs）日益应用于招聘流程，由人工智能引发的偏见问题日益突出。本研究考察了Claude 3.5 Sonnet、GPT-4o、Gemini 1.5和Llama 3.1 405B在生成候选人面试报告时的偏见表现，重点关注性别、种族和年龄等特征。评估了基于LLM的去标识化在缓解此类偏见方面的有效性。结果表明，尽管去标识化能有效降低部分偏见（尤其是性别偏见），但其效果在不同模型和偏见类型间存在差异。值得注意的是，Llama 3.1 405B表现出最低的整体偏见水平。此外，本研究提出的对比去标识化与非去标识化数据的方法，为评估LLM中固有偏见提供了超越招聘场景的新范式。研究强调了谨慎选择模型的重要性，并提出了减少AI应用中偏见的最佳实践，以促进公平与包容。
原文摘要 · Abstract (English)
As large language models (LLMs) become integral to recruitment processes, concerns about AI-induced bias have intensified. This study examines biases in candidate interview reports generated by Claude 3.5 Sonnet, GPT-4o, Gemini 1.5, and Llama 3.1 405B, focusing on characteristics such as gender, race, and age. We evaluate the effectiveness of LLM-based anonymization in reducing these biases. Findings indicate that while anonymization reduces certain biases, particularly gender bias, the degree of effectiveness varies across models and bias types. Notably, Llama 3.1 405B exhibited the lowest overall bias. Moreover, our methodology of comparing anonymized and non-anonymized data reveals a novel approach to assessing inherent biases in LLMs beyond recruitment applications. This study underscores the importance of careful LLM selection and suggests best practices for minimizing bias in AI applications, promoting fairness and inclusivity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。