构建全球礼仪语料库，揭示大模型对不同地区礼仪的偏见。
EtiCor++: Towards Understanding Etiquettical Bias in LLMs
- 构建EtiCor++全球礼仪语料库，覆盖多地区文化规范。
- 实验发现大模型对部分地区存在显著礼仪认知偏见。
- 提供评估工具与指标，适合研究文化敏感性的学者使用。
近年来，研究人员开始关注大模型的文化敏感性。其中，礼仪作为区域特有的文化组成部分，已成为研究热点。由于礼仪具有地域特性且是文化的重要构成，使大模型具备礼仪敏感性至关重要。然而，当前缺乏足够的资源来评估大模型在礼仪理解与偏见方面的表现。本文提出EtiCor++，一个涵盖全球范围的礼仪语料库，设计了多种任务以评估大模型在不同地区礼仪知识上的表现，并引入多种度量指标来检测模型偏差。对多个大模型的广泛实验表明，模型对某些地区存在固有偏见。
原文摘要 · Abstract (English)
In recent years, researchers have started analyzing the cultural sensitivity of LLMs. In this respect, Etiquettes have been an active area of research. Etiquettes are region-specific and are an essential part of the culture of a region; hence, it is imperative to make LLMs sensitive to etiquettes. However, there needs to be more resources in evaluating LLMs for their understanding and bias with regard to etiquettes. In this resource paper, we introduce EtiCor++, a corpus of etiquettes worldwide. We introduce different tasks for evaluating LLMs for knowledge about etiquettes across various regions. Further, we introduce various metrics for measuring bias in LLMs. Extensive experimentation with LLMs shows inherent bias towards certain regions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。