测试大模型在词语义消歧上的表现，发现微调后中等参数模型更优。
Exploring the Word Sense Disambiguation Capabilities of Large Language Models
- 设计两个适配大模型的词义消歧新任务
- 零样本下大模型表现良好，但未超现有最优方法
- 微调后的中等规模模型超越所有其他模型
词语义消歧（WSD）是计算语言学中的经典任务。随着大语言模型（LLM）的兴起，该任务的关注度下降。本文评估多种LLM在WSD上的表现，扩展了此前的基准XL-WSD，重新设计两个适合LLM的任务：1）给定句子中的词语，生成其正确释义；2）给定词语及一组预定义含义，选择正确释义。新基准基于XL-WSD与BabelNet构建。结果表明，LLM在零样本学习中表现良好，但尚未超越当前最优方法。然而，经过微调的中等参数量模型性能优于所有其他模型，包括现有最优方法。
原文摘要 · Abstract (English)
Word Sense Disambiguation (WSD) is a historical task in computational linguistics that has received much attention over the years. However, with the advent of Large Language Models (LLMs), interest in this task (in its classical definition) has decreased. In this study, we evaluate the performance of various LLMs on the WSD task. We extend a previous benchmark (XL-WSD) to re-design two subtasks suitable for LLM: 1) given a word in a sentence, the LLM must generate the correct definition; 2) given a word in a sentence and a set of predefined meanings, the LLM must select the correct one. The extended benchmark is built using the XL-WSD and BabelNet. The results indicate that LLMs perform well in zero-shot learning but cannot surpass current state-of-the-art methods. However, a fine-tuned model with a medium number of parameters outperforms all other models, including the state-of-the-art.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。