评测大模型提取语义框架论元的能力，发现格式与微调可显著提升效果。
Can LLMs Extract Frame-Semantic Arguments?
- 用JSON格式输入能大幅提升模型表现
- 780亿参数模型表现最优，但小模型经微调也接近顶尖水平
- 新方法利用预测的框架元素提升歧义目标识别，适合需要高精度的应用
框架语义解析是自然语言理解的关键任务，但大语言模型（LLMs）在提取框架语义论元方面的能力仍缺乏系统研究。本文对从0.5B到78B参数的多类模型进行了全面评估，考察输入表示格式、模型架构及对未见和域外样本的泛化能力。实验表明，采用基于JSON的输入表示可显著提升性能；虽然更大模型整体表现更优，但经过微调的小模型也能达到具有竞争力的结果。我们还提出一种新方法，通过利用预测的框架元素实现对歧义目标的识别，达到当前最优性能。尽管模型具备较强的泛化能力，但在域外数据上仍存在明显短板。
原文摘要 · Abstract (English)
Frame-semantic parsing is a critical task in natural language understanding, yet the ability of large language models (LLMs) to extract frame-semantic arguments remains underexplored. This paper presents a comprehensive evaluation of LLMs on frame-semantic argument identification, analyzing the impact of input representation formats, model architectures, and generalization to unseen and out-of-domain samples. Our experiments, spanning models from 0.5B to 78B parameters, reveal that JSON-based representations significantly enhance performance, and while larger models generally perform better, smaller models can achieve competitive results through fine-tuning. We also introduce a novel approach to frame identification leveraging predicted frame elements, achieving state-of-the-art performance on ambiguous targets. Despite strong generalization capabilities, our analysis finds that LLMs still struggle with out-of-domain data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。