构建首个针对常见名词的转喻识别数据集,提升NLP对日常语言的理解能力。
ConMeC: A Dataset for Metonymy Resolution with Common Nouns
- 创建包含6000句的转喻数据集ConMeC,标注常见名词是否转喻使用
- 大模型通过思维链提示在标准类别上表现接近监督模型
- 适合研究语义理解、自然语言推理与认知语言学的学者使用
转喻在日常交流中扮演重要角色,人们常通过事物最显著特征或关联概念来指代。例如“公交车今天跳过我们的站点”实际指的是司机做出决定。现有转喻研究多集中于专有名词,但涉及常见名词(如桌子、婴儿、学校)的转喻现象同样普遍且具挑战性。本文提出新数据集ConMeC,包含6000个句子,每句配有一个目标常见名词,并由人工标注其是否在上下文中为转喻用法。同时引入基于思维链的提示方法,用于利用大语言模型检测转喻。我们在ConMeC及另外三个转喻数据集上评估了该方法和一个监督式BERT模型。实验表明,大模型在明确类别上表现可媲美监督模型,但在需细腻语义理解的任务上仍显不足。数据集已公开:https://github.com/SaptGhosh/ConMeC。
原文摘要 · Abstract (English)
Metonymy plays an important role in our daily communication. People naturally think about things using their most salient properties or commonly related concepts. For example, by saying "The bus decided to skip our stop today," we actually mean that the bus driver made the decision, not the bus. Prior work on metonymy resolution has mainly focused on named entities. However, metonymy involving common nouns (such as desk, baby, and school) is also a frequent and challenging phenomenon. We argue that NLP systems should be capable of identifying the metonymic use of common nouns in context. We create a new metonymy dataset ConMeC, which consists of 6,000 sentences, where each sentence is paired with a target common noun and annotated by humans to indicate whether that common noun is used metonymically or not in that context. We also introduce a chain-of-thought based prompting method for detecting metonymy using large language models (LLMs). We evaluate our LLM-based pipeline, as well as a supervised BERT model on our dataset and three other metonymy datasets. Our experimental results demonstrate that LLMs could achieve performance comparable to the supervised BERT model on well-defined metonymy categories, while still struggling with instances requiring nuanced semantic understanding. Our dataset is publicly available at: https://github.com/SaptGhosh/ConMeC.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。