用大模型直接微调，就能达到顶尖语义解析效果。
Evaluation of Finetuned LLMs in AMR Parsing
- 仅微调解码器型大模型，无需复杂结构
- 最高取得SMATCH F1 0.804，接近顶尖模型
- 适合想快速部署高效语义解析的开发者
AMR（抽象语义表示）是一种将句子意义编码为有根、有向无环图的语义形式化方法，节点代表概念，边表示语义关系。本文对四种不同架构的大语言模型（Phi 3.5、Gemma 2、LLaMA 3.2、DeepSeek R1 LLaMA Distilled）进行微调，评估其在LDC2020T02 Gold AMR3.0测试集上的表现。结果表明，仅微调解码器的LLM可达到与复杂前沿（SOTA）AMR解析器相当的性能。其中，LLaMA 3.2在语义准确率上表现突出，实现SMATCH F1 0.804，与APT + Silver（IBM）持平，并接近Graphene Smatch（MBSE）的0.854。分析还发现，LLaMA 3.2在语义性能上领先，而Phi 3.5在结构合理性上更优。
原文摘要 · Abstract (English)
AMR (Abstract Meaning Representation) is a semantic formalism that encodes sentence meaning as rooted, directed, acyclic graphs, where nodes represent concepts and edges denote semantic relations. Finetuning decoder only Large Language Models (LLMs) represent a promising novel straightfoward direction for AMR parsing. This paper presents a comprehensive evaluation of finetuning four distinct LLM architectures, Phi 3.5, Gemma 2, LLaMA 3.2, and DeepSeek R1 LLaMA Distilled using the LDC2020T02 Gold AMR3.0 test set. Our results have shown that straightfoward finetuning of decoder only LLMs can achieve comparable performance to complex State of the Art (SOTA) AMR parsers. Notably, LLaMA 3.2 demonstrates competitive performance against SOTA AMR parsers given a straightforward finetuning approach. We achieved SMATCH F1: 0.804 on the full LDC2020T02 test split, on par with APT + Silver (IBM) at 0.804 and approaching Graphene Smatch (MBSE) at 0.854. Across our analysis, we also observed a consistent pattern where LLaMA 3.2 leads in semantic performance while Phi 3.5 excels in structural validity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。