首个斯里兰卡僧伽罗语历时语料库，覆盖1500年语言演变研究
SiDiaC: Sinhala Diachronic Corpus
- 构建涵盖5至20世纪的僧伽罗语文本语料库，含46部作品5.8万词
- 通过光学识别与后处理实现古籍数字化，支持多层级文本分类
- 适合语言学、历史文献与低资源语言NLP研究者使用
SiDiaC是首个全面的僧伽罗语历时语料库，涵盖公元5世纪至20世纪的历史跨度。语料库包含46部文学作品，共5.8万词，基于出版时间、可得性、作者归属、版权合规及数据署名进行筛选，并经仔细标注。斯里兰卡国家图书馆藏书通过Google Document AI OCR技术数字化，随后进行格式修正与正字法现代化处理。其构建参考了如FarPaHC等语料库的经验，尤其在句法标注与文本规范化策略方面，因共享低资源语言特性。语料库按体裁分为两层：一级为非虚构与虚构二元分类；二级细分为宗教、历史、诗歌、语言学和医学五类。尽管面临稀有文本获取受限及依赖次级日期来源等挑战，SiDiaC仍为僧伽罗语自然语言处理提供基础资源，显著拓展该语言在词汇变迁、新词追踪、历史语法及基于语料的词典编纂方面的研究可能。
原文摘要 · Abstract (English)
SiDiaC, the first comprehensive Sinhala Diachronic Corpus, covers a historical span from the 5th to the 20th century CE. SiDiaC comprises 58k words across 46 literary works, annotated carefully based on the written date, after filtering based on availability, authorship, copyright compliance, and data attribution. Texts from the National Library of Sri Lanka were digitised using Google Document AI OCR, followed by post-processing to correct formatting and modernise the orthography. The construction of SiDiaC was informed by practices from other corpora, such as FarPaHC, particularly in syntactic annotation and text normalisation strategies, due to the shared characteristics of low-resourced language status. This corpus is categorised based on genres into two layers: primary and secondary. Primary categorisation is binary, classifying each book into Non-Fiction or Fiction, while the secondary categorisation is more specific, grouping texts under Religious, History, Poetry, Language, and Medical genres. Despite challenges including limited access to rare texts and reliance on secondary date sources, SiDiaC serves as a foundational resource for Sinhala NLP, significantly extending the resources available for Sinhala, enabling diachronic studies in lexical change, neologism tracking, historical syntax, and corpus-based lexicography.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。