用混合模型提升越南语自然语言推理准确率，最高达82.78%
Transformer-Based Contextualized Language Models Joint with Neural Networks for Natural Language Inference in Vietnamese
- 结合上下文语言模型与神经网络进行联合建模
- XLM-R搭配神经网络在ViNLI上达82.78% F1分数
- 适合资源受限场景下的高效越南语NLP应用
自然语言推理（NLI）是自然语言处理中的重要任务，对多种AI应用具有价值。然而，针对越南语的NLI研究较少，且缺乏对联合模型的探索。本文实验了多种上下文语言模型（CLM）与神经网络的组合方式，利用CLM生成上下文表示，再由神经网络完成分类。在基准数据集ViNLI上，最高F1分数达到82.78%。其中，最大规模的CLM为XLM-R（355M），其表现优于微调PhoBERT（+6.58%）、mBERT（+19.08%）和XLM-R自身（+0.94%）。结果表明，该联合方法简单有效，具备高质量性能，适用于资源高效型应用场景。
原文摘要 · Abstract (English)
Natural Language Inference (NLI) is a task within Natural Language Processing (NLP) that holds value for various AI applications. However, there have been limited studies on Natural Language Inference in Vietnamese that explore the concept of joint models. Therefore, we conducted experiments using various combinations of contextualized language models (CLM) and neural networks. We use CLM to create contextualized work presentations and use Neural Networks for classification. Furthermore, we have evaluated the strengths and weaknesses of each joint model and identified the model failure points in the Vietnamese context. The highest F1 score in this experiment, up to 82.78% in the benchmark dataset (ViNLI). By conducting experiments with various models, the most considerable size of the CLM is XLM-R (355M). That combination has consistently demonstrated superior performance compared to fine-tuning strong pre-trained language models like PhoBERT (+6.58%), mBERT (+19.08%), and XLM-R (+0.94%) in terms of F1-score. This article aims to introduce a novel approach or model that attains improved performance for Vietnamese NLI. Overall, we find that the joint approach of CLM and neural networks is simple yet capable of achieving high-quality performance, which makes it suitable for applications that require efficient resource utilization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。