对比中西方大模型识别残障歧视能力,发现本地模型表现更差且偏见更深。
Disability Across Cultures: A Human-Centered Audit of Ableism in Western and Indic LLMs
- 用印地语翻译歧视数据集,测试8个中西模型的识别能力
- 西方模型误判率高,印度模型低估歧视,均对印地语内容容忍度更高
- 强调应以本地残障者视角重构AI伦理评估标准
残障人士在印度等非西方国家面临严重网络歧视,资源匮乏与根深蒂固的污名化加剧了这一问题。尽管大语言模型被用于识别和缓解网络仇恨,但现有研究多聚焦西方语境与西方模型。本研究将公开的歧视语料库翻译为印地语,让八款模型(四款美国开发:GPT-4、Gemini、Claude、Llama;四款印度开发:Krutrim、Nanda、Gajendra、Airavata)进行评分与解释。同时招募175名来自美印的残障人士完成相同任务。结果显示,西方模型普遍高估歧视程度,而印度模型则普遍低估。更令人担忧的是,所有模型在面对印地语表达时更宽容,且倾向于套用西方歧视框架。而印度残障人士则从意图、关系与韧性出发,强调教育与沟通的意愿。本研究为全球包容性反歧视标准提供基础,强调必须在AI设计与评估中纳入本土残障经验。
原文摘要 · Abstract (English)
People with disabilities (PwD) experience disproportionately high levels of discrimination and hate online, particularly in India, where entrenched stigma and limited resources intensify these challenges. Large language models (LLMs) are increasingly used to identify and mitigate online hate, yet most research on online ableism focuses on Western audiences with Western AI models. Are these models adequately equipped to recognize ableist harm in non-Western places like India? Do localized, Indic language models perform better? To investigate, we adopted and translated a publicly available ableist speech dataset to Hindi, and prompted eight LLMs--four developed in the U.S. (GPT-4, Gemini, Claude, Llama) and four in India (Krutrim, Nanda, Gajendra, Airavata)--to score and explain ableism. In parallel, we recruited 175 PwD from both the U.S. and India to perform the same task, revealing stark differences between groups. Western LLMs consistently overestimated ableist harm, while Indic LLMs underestimated it. Even more concerning, all LLMs were more tolerant of ableism when it was expressed in Hindi and asserted Western framings of ableist harm. In contrast, Indian PwD interpreted harm through intention, relationality, and resilience--emphasizing a desire to inform and educate perpetrators. This work provides groundwork for global, inclusive standards of ableism, demonstrating the need to center local disability experiences in the design and evaluation of AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。