arXiv:2411.13173cs.IRcs.AI2024-11被引 19

发现文本嵌入模型对写作风格存在偏见,影响信息检索公平性。

Writing Style Matters: An Examination of Bias and Fairness in Information Retrieval Systems

  • 分析不同嵌入模型对文档和查询写作风格的偏好差异。
  • 多数模型不青睐非正式、情绪化风格,导致表达受限。
  • 适合关注AI公平性、检索系统设计的研究者阅读。

语言模型技术的快速发展带来了新机遇,也引发了偏见与公平性问题。本文探究了当前通用文本嵌入模型在信息检索(IR)系统中对特定文档和查询写作风格的潜在偏见。研究发现,不同嵌入模型对文档写作风格存在差异偏好,多数模型更不青睐非正式和情感化风格。对于查询风格,许多模型倾向于匹配查询与检索文档的风格,但部分模型表现出对特定风格的一致偏好。在合成数据上微调的嵌入模型,对生成数据的特定风格表现出稳定偏好。这些偏见可能无意中压制或边缘化某些表达方式,威胁信息检索的公平性。此外,我们还比较了基于不同大模型的检索增强生成(RAG)系统答案风格,发现多数嵌入模型在评估答案正确性时,偏向于大模型的输出风格。本研究揭示了基于写作风格的偏见问题,为构建更公平、鲁棒的模型提供了重要启示。

原文摘要 · Abstract (English)

The rapid advancement of Language Model technologies has opened new opportunities, but also introduced new challenges related to bias and fairness. This paper explores the uncharted territory of potential biases in state-of-the-art universal text embedding models towards specific document and query writing styles within Information Retrieval (IR) systems. Our investigation reveals that different embedding models exhibit different preferences of document writing style, while more informal and emotive styles are less favored by most embedding models. In terms of query writing styles, many embedding models tend to match the style of the query with the style of the retrieved documents, but some show a consistent preference for specific styles. Text embedding models fine-tuned on synthetic data generated by LLMs display a consistent preference for certain style of generated data. These biases in text embedding based IR systems can inadvertently silence or marginalize certain communication styles, thereby posing a significant threat to fairness in information retrieval. Finally, we also compare the answer styles of Retrieval Augmented Generation (RAG) systems based on different LLMs and find out that most text embedding models are biased towards LLM's answer styles when used as evaluation metrics for answer correctness. This study sheds light on the critical issue of writing style based bias in IR systems, offering valuable insights for the development of more fair and robust models.

信息检索模型偏见公平性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。