用因果语言模型生成语义框架嵌入,无需大量标注也能高效识别框架。
FrameEOL: Semantic Frame Induction using Causal Language Models
- 基于提示学习和上下文学习,从GPT类模型提取框架嵌入
- 日语数据上仅用5个示例即达MLM微调效果
- 适合资源匮乏语言的语义框架识别任务
语义框架归纳旨在根据词语唤起的语义框架对词语进行聚类。近年来,利用BERT等掩码语言模型(MLM)获得的词嵌入在该任务中表现优异。尽管因果语言模型(CLM,如GPT、Llama系列)在多种语言理解任务中表现出色且能模拟框架理解,但尚未用于语义框架归纳。本文提出一种基于CLM的新方法——FrameEOL,通过提示学习获取以一个框架名作为标签的框架嵌入。为获得更适合框架归纳的嵌入,引入上下文学习(ICL)与深度度量学习(DML)。最终通过聚类生成框架。在英语和日语FrameNet数据集上的实验表明,所提方法优于现有方法。尤其在缺乏丰富资源的日语中,仅使用5个ICL示例的CLM方法性能可媲美经DML微调的MLM方法。
原文摘要 · Abstract (English)
Semantic frame induction is the task of clustering frame-evoking words according to the semantic frames they evoke. In recent years, leveraging embeddings of frame-evoking words that are obtained using masked language models (MLMs) such as BERT has led to high-performance semantic frame induction. Although causal language models (CLMs) such as the GPT and Llama series succeed in a wide range of language comprehension tasks and can engage in dialogue as if they understood frames, they have not yet been applied to semantic frame induction. We propose a new method for semantic frame induction based on CLMs. Specifically, we introduce FrameEOL, a prompt-based method for obtaining Frame Embeddings that outputs One frame-name as a Label representing the given situation. To obtain embeddings more suitable for frame induction, we leverage in-context learning (ICL) and deep metric learning (DML). Frame induction is then performed by clustering the resulting embeddings. Experimental results on the English and Japanese FrameNet datasets demonstrate that the proposed methods outperform existing frame induction methods. In particular, for Japanese, which lacks extensive frame resources, the CLM-based method using only 5 ICL examples achieved comparable performance to the MLM-based method fine-tuned with DML.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。