arXiv:2504.05214cs.CL2025-04

用大模型+记忆回放实现动态关系抽取,避免旧知识遗忘。

Post-Training Language Models for Continual Relation Extraction

  • 采用大语言模型与记忆回放机制,持续学习新关系
  • 在TACRED和FewRel上达到领先性能,尤其Mistral和Flan-T5表现优异
  • 适合需要实时更新知识图谱的系统开发者

真实世界数据如新闻、社交媒体和聊天记录具有动态非平稳特性,给构建实时结构化知识图谱带来挑战。关系抽取(RE)作为知识图谱构建的核心环节,传统模型依赖静态过时数据,难以适应数据演变。持续关系抽取(CRE)通过增量学习新关系并保留已有知识来应对该问题。本研究探索预训练语言模型(特别是大语言模型,LLMs)在CRE中的应用,重点利用记忆回放缓解灾难性遗忘。我们在TACRED和FewRel数据集上评估了Decoder-only模型(如Mistral-7B、Llama2-7B)和Encoder-decoder模型(如Flan-T5 Base)。任务增量微调的LLMs在TACRED上表现优于早期基于BERT等编码器-仅模型的方法,尤其在已见任务准确率和整体性能(全量与平均准确率)上更优,其中Mistral和Flan-T5表现突出。FewRel上的结果同样积极,全量与平均准确率均位列第二。该工作揭示了知识迁移、模型架构与知识图谱完整性在持续关系抽取中的关键作用,推动了基于大模型与记忆回放的动态实时关系抽取技术发展。

原文摘要 · Abstract (English)

Real-world data, such as news articles, social media posts, and chatbot conversations, is inherently dynamic and non-stationary, presenting significant challenges for constructing real-time structured representations through knowledge graphs (KGs). Relation Extraction (RE), a fundamental component of KG creation, often struggles to adapt to evolving data when traditional models rely on static, outdated datasets. Continual Relation Extraction (CRE) methods tackle this issue by incrementally learning new relations while preserving previously acquired knowledge. This study investigates the application of pre-trained language models (PLMs), specifically large language models (LLMs), to CRE, with a focus on leveraging memory replay to address catastrophic forgetting. We evaluate decoder-only models (eg, Mistral-7B and Llama2-7B) and encoder-decoder models (eg, Flan-T5 Base) on the TACRED and FewRel datasets. Task-incremental fine-tuning of LLMs demonstrates superior performance over earlier approaches using encoder-only models like BERT on TACRED, excelling in seen-task accuracy and overall performance (measured by whole and average accuracy), particularly with the Mistral and Flan-T5 models. Results on FewRel are similarly promising, achieving second place in whole and average accuracy metrics. This work underscores critical factors in knowledge transfer, language model architecture, and KG completeness, advancing CRE with LLMs and memory replay for dynamic, real-time relation extraction.

持续学习关系抽取大模型知识图谱

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。