评测大模型对瑞典语多义词消歧能力，发现用人工定义提升效果
How Well Do Large Language Models Disambiguate Swedish Words?
- 在提示中加入人工定义的词义，显著提升消歧准确率
- 现有大模型在有训练数据时仍不如最优监督模型
- 大模型整体优于无监督图方法，适合资源有限场景
我们评估了一系列近期的大语言模型在瑞典语词汇意义消歧两个基准上的表现。目前所有模型在有训练数据时的准确率均低于最佳监督消歧器，但多数模型优于基于图的无监督系统。比较了多种提示方法，重点分析如何在上下文中表达可能的词义。当提示中包含人工编写的词义定义时，取得最佳准确率。
原文摘要 · Abstract (English)
We evaluate a battery of recent large language models on two benchmarks for word sense disambiguation in Swedish. At present, all current models are less accurate than the best supervised disambiguators in cases where a training set is available, but most models outperform graph-based unsupervised systems. Different prompting approaches are compared, with a focus on how to express the set of possible senses in a given context. The best accuracies are achieved when human-written definitions of the senses are included in the prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。