用大模型检测文本中的主观偏见，识别三类隐性偏差。
Subjective Multi-Bias Detection with Large Language Models
- 构建多类型偏见分类框架，涵盖表述、认知和人口统计偏差。
- 在超过4000对维基编辑语句中实现三类偏见的精准识别。
- 适合关注内容安全与公平性的研究人员及AI伦理实践者。
本项目聚焦文本中普遍存在的偏见检测问题，尤其关注主观偏见——即引入不当态度或与事实不符的表述。此类偏见会损害文本的真实性与可信度，引发误解甚至社会矛盾，尤其当涉及攻击性语言时更为显著。受先前工作启发，我们针对三类主观偏见展开研究：(1) 表述偏见，即使用带有特定立场的一边倒词汇；(2) 认知偏见，包含影响文本可信度的微妙语言特征；(3) 人口统计偏见，基于性别或宗教等预设因素的词汇/短语使用。实验基于维基百科编辑语料库WIKIBIAS，包含4000余对句子，每对标注为四类之一：表述偏见、认知偏见、人口统计偏见或无偏见。目标是实现对多跨度偏见的准确分类与标注。项目代码已开源至https://github.com/HoningJade/LLM-Bias-Type-Classification。
原文摘要 · Abstract (English)
In this project, we delved into the pervasive challenge of bias detection within the text content. More specifically, our focus lies on the identification of subjective bias, a type of bias that introduces improper attitudes or portrays a statement at odds with the actual truth. The subjective bias can jeopardize the authenticity and reliability of texts, leading to misconceptions and potential social tensions, especially when expressed through offensive language. Following prior work [1], we tackled with three different types of subjective biases in text: (1) framing bias with the use of one-sided words or phrases containing a particular point of view; (2) epistemological bias which includes subtle linguistic features that can affect the believability of the texts; (3) demographic bias with word/phrase usage under presuppositions of a particular demographic factor (i.e., gender or religion). In terms of the data we utilize, the input consists of texts that may harbor subjective biases. The output is a classification or annotation that reveals the presence or absence of such biases within the provided content. More specifically, we detected three different types of multi-span biases in corpus WIKIBIAS [2] with more than 4,000 sentence pairs from Wikipedia edits. The data is labelled by bias type for span pairs with the following categories: (1) framing bias, (2) epistemological bias, (3) demographic bias, and (4) no bias. The project codes are released at https://github.com/HoningJade/LLM-Bias-Type-Classification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。