arXiv:2608.24698cs.CL2026-08

伊朗波斯语并非整体缺数据,而是标注资源分布不均。

The Annotation Bottleneck in Persian Text NLP: Persian as an Annotation-Scarce Language

  • 将波斯语标注稀缺性归因于资源生态不均,而非语言本身缺陷。
  • 跨任务对比显示,语法与新闻实体识别标注较密,自然语言推理则严重不足。
  • 适合关注低资源语言标注分布、多领域标注差异的研究者。

波斯语常被视作自然语言处理中的低资源语言,但这一标签掩盖了不同层面的短缺。本文主张,更准确的说法是波斯语为标注稀缺语言,前提是将该属性理解为NLP资源生态特征,而非语言固有属性。论文综述了截至2026年7月可用的34个代表性波斯语文本资源,并进行三项定量验证:第一,独立网络测量显示波斯语在约20种最可见内容语言中位居前列——W3Techs报告其占已知语言网站的0.9%,Common Crawl CC-MAIN-2026-30确认其为主语言的网页占比0.7039%;第二,语音资源从FARSDAT到近年含数百至数千小时语音的语料库,呈现长期发展轨迹;第三,通过以公共网络存在度为基准的波斯语-英语任务标注量匹配比较,发现标注密度差异显著:语法与新闻命名实体识别相对密集,而自然语言推理低于按网页比例应有水平。证据表明,波斯语并非整体标注匮乏,而是标注稀缺表现为任务与领域覆盖不均、标注体系不兼容、访问与文档障碍,以及对专业领域、偏好数据及非标准伊朗波斯语变体的监督不足。

原文摘要 · Abstract (English)

Persian (Farsi) is often described as a low-resource language in natural language processing, but that label collapses distinct shortages into a single category. This paper argues that Persian is more precisely described as annotation-scarce, provided that the term is understood as a property of its NLP resource ecology rather than an intrinsic property of the language. The review covers 34 representative Persian text resources available by July 2026 and adds three quantitative cross-checks. First, independent web measurements place Persian among roughly the twenty most visible content languages: W3Techs reports Persian on about 0.9% of websites with a known content language, while Common Crawl CC-MAIN-2026-30 identifies Persian as the primary language of 0.7039% of HTML pages. Second, a selective speech review shows a long resource trajectory from FARSDAT to recent corpora containing hundreds or thousands of hours of speech. Third, a matched Persian-English comparison normalizes task-specific annotation volumes by relative Common Crawl web presence. The resulting ratios vary sharply: Persian syntax and news NER are comparatively dense, whereas natural-language inference falls below the web-proportional baseline. The evidence therefore does not support a simple claim that Persian is globally deficient in labeled volume. Instead, annotation scarcity is expressed through uneven task and domain coverage, incompatible schemes, access and documentation friction, and limited supervision for specialist domains, preference data, and varieties beyond standard Iranian Persian.

低资源语言标注稀缺波斯语NLP生态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。