构建首个泰尼迪语音节划分数据集并用深度学习实现99.21%准确率。
Tenyidie Syllabification corpus creation and deep learning applications
- 创建10,120个音节标注的泰尼迪语数据集，用于深度学习训练。
- BLSTM模型在测试集上达到99.21%准确率，表现最佳。
- 为低资源语言的形态分析、机器翻译等任务提供基础支持。
泰尼迪语是印度东北部那加兰邦特尼米亚族群使用的藏缅语系低资源语言，具有声调、主宾谓语序和高度黏着性特征。目前尚无该语言音节划分的研究。本文首次构建了包含10,120个音节化单词的数据集，并应用LSTM、BLSTM、BLSTM+CRF和编码器-解码器四种深度学习架构进行实验。在80:10:10的训练/验证/测试集划分下，BLSTM模型在测试集上达到99.21%的最高准确率。该成果可广泛应用于泰尼迪语的形态分析、词性标注、机器翻译等自然语言处理任务。
原文摘要 · Abstract (English)
The Tenyidie language is a low-resource language of the Tibeto-Burman family spoken by the Tenyimia Community of Nagaland in the north-eastern part of India and is considered a major language in Nagaland. It is tonal, Subject-Object-Verb, and highly agglutinative in nature. Being a low-resource language, very limited research on Natural Language Processing (NLP) has been conducted. To the best of our knowledge, no work on syllabification has been reported for this language. Among the many NLP tasks, syllabification or syllabication is an important task in which the given word syllables are identified. The contribution of this work is the creation of 10,120 syllabified Tenyidie words and the application of the Deep Learning techniques on the created corpus. In this paper, we have applied LSTM, BLSTM, BLSTM+CRF, and Encoder-decoder deep learning architectures on our created dataset. In our dataset split of 80:10:10 (train:validation:test) set, we achieved the highest accuracy of 99.21% with BLSTM model on the test set. This work will find its application in numerous other NLP applications, such as morphological analysis, part-of-speech tagging, machine translation, etc, for the Tenyidie Language. Keywords: Tenyidie; NLP; syllabification; deep learning; LSTM; BLSTM; CRF; Encoder-decoder
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。