AI偏见会悄悄影响人类决策,即使用户觉得AI不靠谱也难逃其影响。
No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy
- 用模拟AI偏见测试人类在招聘中的行为变化。
- 当AI偏袒某族裔时,人选择该族裔候选人高达90%。
- 提前做隐性偏见测试可降低13%的非刻板印象选择概率。
本研究开展了一项简历筛选实验(N=528),参与者与模拟具有种族偏好(偏见)的AI模型协作,评估16种高低声望职业的候选人。模拟的AI偏见基于真实AI系统中种族偏见的事实与反事实估计。研究考察了人们对白人、黑人、西班牙裔和亚裔候选人的偏好(通过姓名和认同群体呈现于质量控制简历上),共涵盖1,526种情景,并使用内隐联想测试(IAT)测量其无意识的种族与地位关联,该关联可预测歧视性招聘决策,但此前未在人机协作中被研究。在无AI或无种族偏见的AI辅助下,人们平等选择各类候选人;但当与偏袒特定群体的AI互动时,人们也会随之偏袒该群体,选择率达90%。若参与者在简历筛选前完成IAT测试,其选择不符合常见种族-地位刻板印象候选人的可能性将提升13%。此外,即使人们认为AI建议质量低或无关紧要,其决策仍可能受AI偏见影响。该研究对人机协同决策中的人类自主性、工作场景下的AI应用、以及偏差缓解策略具有重要意义,提示组织与监管政策应充分认识人机协作决策的复杂性,加强对使用者的教育,并明确需监督的系统范围。
原文摘要 · Abstract (English)
In this study, we conduct a resume-screening experiment (N=528) where people collaborate with simulated AI models exhibiting race-based preferences (bias) to evaluate candidates for 16 high and low status occupations. Simulated AI bias approximates factual and counterfactual estimates of racial bias in real-world AI systems. We investigate people's preferences for White, Black, Hispanic, and Asian candidates (represented through names and affinity groups on quality-controlled resumes) across 1,526 scenarios and measure their unconscious associations between race and status using implicit association tests (IATs), which predict discriminatory hiring decisions but have not been investigated in human-AI collaboration. When making decisions without AI or with AI that exhibits no race-based preferences, people select all candidates at equal rates. However, when interacting with AI favoring a particular group, people also favor those candidates up to 90% of the time, indicating a significant behavioral shift. The likelihood of selecting candidates whose identities do not align with common race-status stereotypes can increase by 13% if people complete an IAT before conducting resume screening. Finally, even if people think AI recommendations are low quality or not important, their decisions are still vulnerable to AI bias under certain circumstances. This work has implications for people's autonomy in AI-HITL scenarios, AI and work, design and evaluation of AI hiring systems, and strategies for mitigating bias in collaborative decision-making tasks. In particular, organizational and regulatory policy should acknowledge the complex nature of AI-HITL decision making when implementing these systems, educating people who use them, and determining which are subject to oversight.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。