从机制层面解析大模型中的偏见，发现偏见集中在少数层且影响其他任务。
Dissecting Bias in LLMs: A Mechanistic Interpretability Perspective
- 通过分析模型内部连接，定位导致性别与族裔偏见的关键神经元
- 偏见计算高度局部化，仅存在于少量模型层中
- 移除偏见组件会连带影响命名实体识别等下游任务
大型语言模型（LLMs）因训练数据而表现出社会、人口和性别偏见。本文采用机制可解释性方法，分析GPT-2和Llama2等模型中此类偏见的结构表征。聚焦于人口与性别偏见，我们通过多种指标识别出导致偏见行为的内部连接。进一步评估这些组件在不同数据集和语言变化下的稳定性、定位性和泛化能力。系统消融实验表明，偏见相关计算高度局部化，通常集中于少数层。此外，这些组件在不同微调设置下会发生变化，包括与偏见无关的设置。最后，我们证明移除这些组件不仅能降低偏见输出，还会对命名实体识别和语言可接受性判断等任务产生影响，因为这些关键组件与其他任务共享。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are known to exhibit social, demographic, and gender biases, often as a consequence of the data on which they are trained. In this work, we adopt a mechanistic interpretability approach to analyze how such biases are structurally represented within models such as GPT-2 and Llama2. Focusing on demographic and gender biases, we explore different metrics to identify the internal edges responsible for biased behavior. We then assess the stability, localization, and generalizability of these components across dataset and linguistic variations. Through systematic ablations, we demonstrate that bias-related computations are highly localized, often concentrated in a small subset of layers. Moreover, the identified components change across fine-tuning settings, including those unrelated to bias. Finally, we show that removing these components not only reduces biased outputs but also affects other NLP tasks, such as named entity recognition and linguistic acceptability judgment because of the sharing of important components with these tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。