arXiv:2410.23133cs.CL2024-10被引 4

通过众包识别跨语言词汇空缺,提升多语种词典的多样性与公平性。

Crowdsourcing Lexical Diversity

  • 设计微任务平台,让工人对比双语词汇差异
  • 在食物领域发现2140个英阿词汇空缺,951个印尼语-班贾尔语空缺
  • 适合语言保护、文化研究者及多语NLP开发者使用

词汇语义资源(LSRs)如在线词典和词网,是自然语言处理以及语言人类学、语言保护等领域的重要基础。然而,许多语言的这类资源存在质量缺陷:包含错误条目、内容不全,更严重的是对英语和盎格鲁-撒克逊文化的偏见。这种偏见体现在特定语言或文化概念缺失、引入外来(盎格鲁-撒克逊)概念,以及未明确标注术语在另一语言中不可译的情况,即跨语言词汇空缺。本文提出一种新的众包方法以减少此类偏见。通过众包工作者在词汇多样性丰富的领域(如亲属关系、食物)对比两种语言的词素,利用LingoGap众包平台进行微任务,识别等价词、语言特有词及词汇空缺。我们在两个案例研究中验证了该方法:(1) 英语与阿拉伯语,(2) 标准印度尼西亚语与班贾尔语。实验分别识别出2,140个和951个词汇空缺。结果证实该方法与工具在大规模词典扩充任务中的可用性。

原文摘要 · Abstract (English)

Lexical-semantic resources (LSRs), such as online lexicons and wordnets, are fundamental to natural language processing applications as well as to fields such as linguistic anthropology and language preservation. In many languages, however, such resources suffer from quality issues: incorrect entries, incompleteness, but also the rarely addressed issue of bias towards the English language and Anglo-Saxon culture. Such bias manifests itself in the absence of concepts specific to the language or culture at hand, the presence of foreign (Anglo-Saxon) concepts, as well as in the lack of an explicit indication of untranslatability, also known as cross-lingual lexical gaps, when a term has no equivalent in another language. This paper proposes a novel crowdsourcing methodology for reducing bias in LSRs. Crowd workers compare lexemes from two languages, focusing on domains rich in lexical diversity, such as kinship or food. Our LingoGap crowdsourcing platform facilitates comparisons through microtasks identifying equivalent terms, language-specific terms, and lexical gaps across languages. We validated our method by applying it to two case studies focused on food-related terminology: (1) English and Arabic, and (2) Standard Indonesian and Banjarese. These experiments identified 2,140 lexical gaps in the first case study and 951 in the second. The success of these experiments confirmed the usability of our method and tool for future large-scale lexicon enrichment tasks.

众包词汇空缺语言公平多语种

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。