将语法框架UD融入预训练模型，显著提升跨语言句子辨识性能。
Evaluating the Effectiveness of Linguistic Knowledge in Pretrained Language Models: A Case Study of Universal Dependencies
- 在预训练模型中引入通用语法框架UD，增强语言理解能力。
- 跨语言对抗性同义句识别准确率提升3.85%，F1值提高6.08%。
- 适合关注多语言模型泛化与语法知识利用的研究者。
通用依存关系（Universal Dependencies, UD）虽被广泛认为是跨语言句法表征最成功的框架，但其实际有效性仍缺乏深入探索。本文通过将UD融入预训练语言模型，评估其在跨语言对抗性同义句识别任务中的表现。实验结果显示，引入UD后，模型准确率与F1值平均分别提升3.85%和6.08%。该改进缩小了预训练模型与大语言模型在部分语种对上的差距，甚至在某些情况下超越后者。此外，某语言与英语的UD相似度与其模型性能呈正相关。两项发现均证明了UD在跨领域任务中的有效性与潜力。
原文摘要 · Abstract (English)
Universal Dependencies (UD), while widely regarded as the most successful linguistic framework for cross-lingual syntactic representation, remains underexplored in terms of its effectiveness. This paper addresses this gap by integrating UD into pretrained language models and assesses if UD can improve their performance on a cross-lingual adversarial paraphrase identification task. Experimental results show that incorporation of UD yields significant improvements in accuracy and $F_1$ scores, with average gains of 3.85\% and 6.08\% respectively. These enhancements reduce the performance gap between pretrained models and large language models in some language pairs, and even outperform the latter in some others. Furthermore, the UD-based similarity score between a given language and English is positively correlated to the performance of models in that language. Both findings highlight the validity and potential of UD in out-of-domain tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。