发现主流数据集目录低估了多语种研究中的真实数据存在,许多大语种其实已有研究用数据。
Beyond Catalogue Counts: the Dataset Visibility Asymmetry in Low-Resource Multilingual NLP
- 通过文献挖掘发现141个低可见性语言中存在609个实际研究用数据集
- 59%的大语种在目录中记录为零数据,但实际有研究活动
- 强调数据可发现性和长期可访问性比单纯数量更重要
多语种自然语言处理常依赖中心化目录的数据集数量来判断语言资源丰缺。然而这些目录仅记录已注册或机构分发的数据,未必反映实际被创建、引用或重用的状况。本文结合目录数据与文献证据,提出资源密度指数(RDI),计算200种最广泛使用语言的每百万讲者拥有的目录数据集数。其中118种语言(59%)在LRE Map和LDC中平均RDI为零,另23种低于0.1(即每千万讲者最多一个目录数据集)。随后,我们对这141种低可见性语言采用基于LLM的引文挖掘流程,在Semantic Scholar语料库中识别出609个独立数据集,覆盖53种语言,其中356个仍可通过有效公开链接获取。结果揭示显著的可见性差距:许多大语种在目录中看似数据贫乏,但在研究文献中已有明确数据活动。研究提示,多语种数据稀缺不仅是生产问题,更是文档化、可发现性与长期可访问性的挑战。代码与数据已公开于https://github.com/zhiyintan/dataset-visibility-asymmetry。
原文摘要 · Abstract (English)
Multilingual NLP often relies on dataset counts from centralized catalogues to characterize which languages are resource-rich or resource-poor. However, these catalogues record only one layer of dataset visibility: what has been registered or institutionally distributed. They do not necessarily reflect which datasets are created, cited, or reused in the research literature. To examine this gap, we combine a catalogue-based baseline with literature-backed evidence of dataset circulation. We introduce the Resource Density Index (RDI), defined as the number of catalogued datasets per one million speakers, and compute it for the 200 most widely spoken languages in Ethnologue. Among them, 118 languages (59%) have an average RDI of zero across the LRE Map and the Linguistic Data Consortium (LDC), and another 23 fall below 0.1, corresponding to at most one catalogued dataset per ten million speakers. We then apply an LLM-assisted citation-mining pipeline over the Semantic Scholar corpus to these 141 low-visibility languages. After manual validation and consolidation, we identify 609 unique datasets across 53 languages, of which 356 remain openly accessible through working public links. These results reveal a substantial visibility gap: many large-speaker languages appear data-poor in catalogue records yet show clear evidence of dataset activity in the research literature. Our findings suggest that multilingual data scarcity should be understood not only as a production problem, but also as a question of documentation, discoverability, and long-term accessibility. Code and data are publicly available at (https://github.com/zhiyintan/dataset-visibility-asymmetry).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。