arXiv:2602.22200cs.CL2026-02被引 1

构建首个苏美尔泥板文字与转写匹配数据集,推动AI辅助古文字研究。

SumTablets: A Transliteration Dataset of Sumerian Tablets

  • 将9.16万块苏美尔泥板的楔形文字与转写对齐,用特殊标记保留结构信息。
  • 基于Transformer模型实现97.55%字符级准确率,显著提升转写效率。
  • 开源数据集和代码,适合古文字学与NLP交叉研究者使用。

苏美尔转写是学者用拉丁字母表示泥板解读的传统体系。得益于ETCSL、CDLI和Oracc等数字亚述学项目,大量苏美尔转写已在线发布,且结构化良好,适用于多种搜索与分析任务。然而,缺乏一个全面、可访问的数据集将转写与泥板的楔形文字数字表示配对,阻碍了现代自然语言处理(NLP)方法在苏美尔转写中的应用。为填补这一空白,我们提出SumTablets,该数据集将91,606块苏美尔楔形文字泥板的Unicode表示(共6,970,407个字形)与Oracc发布的转写进行配对。通过预处理和标准化转写,并将每个读音映射回源字形的Unicode表示来构建数据集。同时,利用特殊标记保留平行结构信息(如表面、换行、断裂段)。我们以Hugging Face Dataset形式发布SumTablets(CC BY 4.0),并开源数据准备代码。此外,我们基于SumTablets实现了两种转写基线:(1) 基于字形可能读音的加权采样,(2) 微调自回归语言模型。微调后的语言模型达到平均97.55%的字符级F分数(chrF),证明基于Transformer的转写模型在帮助专家快速验证生成转写方面具有即时潜力,而非逐块手动转写。

原文摘要 · Abstract (English)

Sumerian transliteration is a conventional system for representing a scholar's interpretation of a tablet in the Latin script. Thanks to visionary digital Assyriology projects such as ETCSL, CDLI, and Oracc, a large number of Sumerian transliterations have been published online, and these data are well-structured for a variety of search and analysis tasks. However, the absence of a comprehensive, accessible dataset pairing transliterations with a digital representation of the tablet's cuneiform glyphs has prevented the application of modern Natural Language Processing (NLP) methods to the task of Sumerian transliteration. To address this gap, we present SumTablets, a dataset pairing Unicode representations of 91,606 Sumerian cuneiform tablets (totaling 6,970,407 glyphs) with the associated transliterations published by Oracc. We construct SumTablets by first preprocessing and standardizing the Oracc transliterations before mapping each reading back to the Unicode representation of the source glyph. Further, we retain parallel structural information (e.g., surfaces, newlines, broken segments) through the use of special tokens. We release SumTablets as a Hugging Face Dataset (CC BY 4.0) and open source data preparation code via GitHub. Additionally, we leverage SumTablets to implement and evaluate two transliteration baselines: (1) weighted sampling from a glyph's possible readings, and (2) fine-tuning an autoregressive language model. Our fine-tuned language model achieves an average transliteration character-level F-score (chrF) of 97.55, demonstrating the immediate potential of transformer-based transliteration models in allowing experts to rapidly verify generated transliterations rather than manually transliterating tablets one-by-one.

古文字NLP数据集楔形文字

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。