解决塔吉克语与波斯语间因书写系统不同导致的阅读障碍
Connecting the Persian-speaking World through Transliteration
- 基于Transformer的音素到字符转换模型实现跨书写系统转写
- 在新构建的数据集上达到最高74.20的chrF++得分
- 为语言互通提供实用方案，适合多语言互联网接入研究者
尽管塔吉克波斯语、伊朗波斯语和阿富汗波斯语属于同一种语言的不同变体且相互可懂，但因塔吉克语使用改良西里尔字母而其他两国使用波斯-阿拉伯字母，导致塔吉克语使用者无法直接阅读主流波斯语网络内容。由于两种正式语体高度相似且缺乏塔吉克-波斯平行语料，机器转写比机器翻译更可行。本文提出一种基于Transformer的字符到音素（G2P）转写方法，在新构建的双书写系统数据集上实现了58.70（波斯语→塔吉克语）和74.20（塔吉克语→波斯语）的chrF++得分，为未来研究建立可比基准。结果还表明该任务在两个方向均具非平凡难度。论文还概述了两种书写系统的差异及挑战，以支持后续研究。
原文摘要 · Abstract (English)
Despite speaking mutually intelligible varieties of the same language, speakers of Tajik Persian, written in a modified Cyrillic alphabet, cannot read Iranian and Afghan texts written in the Perso-Arabic script. As the vast majority of Persian text on the Internet is written in Perso-Arabic, monolingual Tajik speakers are unable to interface with the Internet in any meaningful way. Due to overwhelming similarity between the formal registers of these dialects and the scarcity of Tajik-Farsi parallel data, machine transliteration has been proposed as more a practical and appropriate solution than machine translation. This paper presents a transformer-based G2P approach to Tajik-Farsi transliteration, achieving chrF++ scores of 58.70 (Farsi to Tajik) and 74.20 (Tajik to Farsi) on novel digraphic datasets, setting a comparable baseline metric for future work. Our results also demonstrate the non-trivial difficulty of this task in both directions. We also provide an overview of the differences between the two scripts and the challenges they present, so as to aid future efforts in Tajik-Farsi transliteration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。