arXiv:2509.02514cs.CLcs.LG2025-09被引 1

对比多种模型在混语命名实体识别中的表现,发现专用模型更优。

Comparative Study of Pre-Trained BERT and Large Language Models for Code-Mixed Named Entity Recognition

  • 用混语数据训练的HingRoBERTa等模型表现最佳。
  • 专用模型在F1-score上显著优于非混语模型和闭源大模型。
  • 适合关注多语言、混语自然语言处理的研究者。

代码混杂文本(尤其是印地-英语混合语)的命名实体识别因结构不规范、音译频繁及语言切换频繁而面临挑战。本研究对比了基于混语数据微调的模型与非混语多语言模型,以及零样本生成式大语言模型的表现。具体评估了在混语数据上训练的HingBERT、HingMBERT和HingRoBERTa,以及在非混语多语言数据上训练的BERT Base Cased、IndicBERT、RoBERTa和MuRIL。还测试了在去除标注的修改版数据集上使用Google Gemini的零样本性能。所有模型均在基准Hinglish NER数据集上以精确率、召回率和F1-score进行评估。结果显示,混语预训练模型(尤其是微调后的HingRoBERTa和HingBERT)表现最优,超越了包括Google Gemini在内的闭源大模型,归因于领域特定预训练。非混语模型表现尚可但适应性有限。值得注意的是,Google Gemini展现出有竞争力的零样本性能,凸显现代大模型的强大泛化能力。本研究为混语命名实体识别中专用与通用模型的有效性提供了关键见解。

原文摘要 · Abstract (English)

Named Entity Recognition (NER) in code-mixed text, particularly Hindi-English (Hinglish), presents unique challenges due to informal structure, transliteration, and frequent language switching. This study conducts a comparative evaluation of code-mixed fine-tuned models and non-code-mixed multilingual models, along with zero-shot generative large language models (LLMs). Specifically, we evaluate HingBERT, HingMBERT, and HingRoBERTa (trained on code-mixed data), and BERT Base Cased, IndicBERT, RoBERTa and MuRIL (trained on non-code-mixed multilingual data). We also assess the performance of Google Gemini in a zero-shot setting using a modified version of the dataset with NER tags removed. All models are tested on a benchmark Hinglish NER dataset using Precision, Recall, and F1-score. Results show that code-mixed models, particularly HingRoBERTa and HingBERT-based fine-tuned models, outperform others - including closed-source LLMs like Google Gemini - due to domain-specific pretraining. Non-code-mixed models perform reasonably but show limited adaptability. Notably, Google Gemini exhibits competitive zero-shot performance, underlining the generalization strength of modern LLMs. This study provides key insights into the effectiveness of specialized versus generalized models for code-mixed NER tasks.

命名实体识别混语处理BERT大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。