arXiv:2605.04576cs.CL2026-05被引 1

首个塔吉克语词性标注基准,对比多种模型表现。

Benchmarking POS Tagging for the Tajik Language: A Comparative Study of Neural Architectures on the TajPersParallel Corpus

论文配图:Benchmarking POS Tagging for the Tajik Language: A Comparative Study of Neural Architectures on the TajPersParallel Corpus
图 1 · 摘自论文原文
  • 在无上下文的词素级别上比较双向LSTM-CRF与多语言Transformer模型。
  • mBERT+LoRA表现最佳,宏观F1为0.11,加权F1达0.62。
  • 塔吉克语与波斯语、俄语在语法结构上最接近,适合跨语言迁移。

本文首次为塔吉克语构建了自动词性标注任务的基准。尽管多语言模型在众多语言中表现出色,但其对塔吉克语的语法分析能力仍未知。本研究通过系统比较经典神经网络架构与现代多语言Transformer模型,填补该空白。实验基于包含约4.4万条目、无完整句子的TajPersParallel语料库,在孤立词素层面进行上下文无关分类,难度极高。比较模型包括双向BiLSTM-CRF,以及XLM-RoBERTa(large)、mBERT、ParsBERT(波斯语)、ruBERT(俄语),均采用参数高效微调方法LoRA。结果表明,mBERT+LoRA表现最优,宏平均F1为0.11,加权F1为0.62。所有模型在缺乏句法上下文时均难以解决形态歧义,仅能准确识别高频词类(如“名词”“形容词”),对罕见功能词完全无效。零样本评估显示塔吉克语与波斯语(ParsBERT)和俄语(ruBERT)在语言类型学上最为接近。研究结果为塔吉克语自动化处理提供了基础。

原文摘要 · Abstract (English)

This paper presents the first benchmark for the task of automatic part-of-speech (POS) tagging for the Tajik language. Despite the existence of multilingual language models demonstrating high effectiveness for many of the world's languages, their capacity for grammatical analysis of Tajik has remained unexplored until now. The aim of this study is to fill this gap through a systematic comparison of classical neural network architectures and modern multilingual transformers. Experiments were conducted on the TajPersParallel corpus, a parallel lexical resource comprising approximately 44,000 dictionary entries. Due to the absence of full-fledged example sentences in the current version of the corpus, the task was performed at the level of isolated lexical units, representing a challenging case of context-independent classification. The study compares the following architectures: a recurrent BiLSTM-CRF model, as well as multilingual models XLM-RoBERTa (large), mBERT, ParsBERT (Persian), and ruBERT (Russian), adapted using the parameter-efficient fine-tuning method LoRA. The testing results showed that the best performance is achieved by the mBERT + LoRA model (macro F1-score = 0.11, weighted F1-score = 0.62). It was established that in the absence of syntactic context, all models experience significant difficulty in resolving morphological ambiguity, successfully classifying primarily high-frequency classes ("noun," "adjective") while demonstrating zero effectiveness for rare function words. Zero-shot evaluation revealed the greatest typological proximity of Tajik to Persian (ParsBERT) and Russian (ruBERT). The obtained results form a foundation for further research and development in the field of automatic processing of the Tajik language.

词性标注多语言模型塔吉克语LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。