复杂句子中，小模型比大模型更擅长关系抽取。
LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs
- 用图结构解析器处理复杂语言结构
- 关系越多，图解析器越优于大模型
- 适合处理复杂语义的场景
关系抽取是构建知识图谱的核心环节。尽管大语言模型（LLMs）在监督和上下文学习中被广泛采用，但本文发现：当文本背后的语言图结构复杂时，其性能仍落后于更轻量的图解析器。我们在六个关系抽取数据集上对比了四个LLMs与图解析器，这些数据集包含不同规模和复杂度的句法图。结果表明，随着输入文档中关系数量的增加，图解析器的表现持续超越LLMs。因此，在复杂语言图存在时，轻量级图解析器成为更优选择。
原文摘要 · Abstract (English)
Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。