基于语义的土耳其语词干化与形态标注模型首次提出
Context Aware Lemmatization and Morphological Tagging Method in Turkish
- 结合拼写与语义信息，用双向LSTM和Turkish BERT建模
- 在IMST与PUD数据集上表现优于SIGMORPHON 2019竞赛结果
- 适合需要精准形态分析的土耳其语自然语言处理任务
词干是定义词语最小单位，能提升多种应用效果。本文提出一种土耳其语词干化与形态标注联合模型，首次考虑词义信息进行词干提取与语法特征预测。模型采用双向LSTM处理拼写，Turkish BERT捕捉语义，并在Universal Dependencies的IMST和PUD数据集上训练。实验结果表明，该模型在词干化与形态标注任务上均超越SIGMORPHON 2019竞赛最优成绩，首次公开了具有语义感知能力的土耳其语词干化方法。
原文摘要 · Abstract (English)
The smallest part of a word that defines the word is called a word root. Word roots are used to increase success in many applications since they simplify the word. In this study, the lemmatization model, which is a word root finding method, and the morphological tagging model, which predicts the grammatical knowledge of the word, are presented. The presented model was developed for Turkish, and both models make predictions by taking the meaning of the word into account. In the literature, there is no lemmatization study that is sensitive to word meaning in Turkish. For this reason, the present study shares the model and the results obtained from the model on Turkish lemmatization for the first time in the literature. In the present study, in the lemmatization and morphological tagging models, bidirectional LSTM is used for the spelling of words, and the Turkish BERT model is used for the meaning of words. The models are trained using the IMST and PUD datasets from Universal Dependencies. The results from the training of the models were compared with the results from the SIGMORPHON 2019 competition. The results of the comparisons revealed that our models were superior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。