大模型对德语方言使用者存在系统性偏见,影响语言判断与决策。
Large Language Models Discriminate Against Speakers of German Dialects
- 通过关联与决策任务检测模型对方言的负面刻板印象。
- 所有模型均表现出显著的方言命名与使用偏见,倾向负面描述。
- 明确标注方言身份会放大偏见,比隐含线索更严重。
方言是全球各地文化的重要组成部分。在德国,超过40%的人口讲地区方言(Adler and Hansen, 2022)。尽管具有文化价值,方言使用者常面临社会负面刻板印象。本文研究大型语言模型(LLMs)是否复制此类偏见。基于社会语言学中关于方言感知的研究,分析与方言使用者相关的常见特质,并在两个任务中评估模型的方言命名偏见与方言使用偏见:关联任务与决策任务。为评估方言使用偏见,构建了一个新评估语料库,包含七种德国方言(如阿勒曼尼语和巴伐利亚语)及其标准德语对应句。结果发现:(1) 所有被评估的LLMs在关联任务中均表现出显著的方言命名与方言使用偏见,体现为负面形容词关联;(2) 所有模型在决策任务中也重现了这些偏见;(3) 与先前研究显示显式人口统计提及偏见较小不同,本文发现显式标注语言身份(如‘德语方言使用者’)反而比隐含线索(如方言使用)放大偏见。
原文摘要 · Abstract (English)
Dialects represent a significant component of human culture and are found across all regions of the world. In Germany, more than 40% of the population speaks a regional dialect (Adler and Hansen, 2022). However, despite cultural importance, individuals speaking dialects often face negative societal stereotypes. We examine whether such stereotypes are mirrored by large language models (LLMs). We draw on the sociolinguistic literature on dialect perception to analyze traits commonly associated with dialect speakers. Based on these traits, we assess the dialect naming bias and dialect usage bias expressed by LLMs in two tasks: an association task and a decision task. To assess a model's dialect usage bias, we construct a novel evaluation corpus that pairs sentences from seven regional German dialects (e.g., Alemannic and Bavarian) with their standard German counterparts. We find that: (1) in the association task, all evaluated LLMs exhibit significant dialect naming and dialect usage bias against German dialect speakers, reflected in negative adjective associations; (2) all models reproduce these dialect naming and dialect usage biases in their decision making; and (3) contrary to prior work showing minimal bias with explicit demographic mentions, we find that explicitly labeling linguistic demographics--German dialect speakers--amplifies bias more than implicit cues like dialect usage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。