用图注意力网络增强大模型对事件时序关系的长距离依赖捕捉能力。
GDLLM: A Global Distance-aware Modeling Approach Based on Large Language Models for Event Temporal Relation Extraction
- 构建距离感知图结构,辅助大模型识别远距离事件关系。
- 在TB-Dense和MATRES数据集上达到当前最优性能,少数类准确率显著提升。
- 适合需要精准处理复杂时序逻辑的任务,如新闻事件分析、医疗记录推理。
在自然语言处理中,事件时序关系抽取(ETRE)旨在识别两个事件间的时序关系。现有研究虽重视语言模型的作用,但小型语言模型因预训练知识有限,难以处理不平衡数据集中少数类关系;而大型语言模型通常依赖人工设计的提示或指令,可能引入噪声,干扰模型对事件间长距离依赖的判断。为此,本文提出基于大语言模型的全局距离感知建模方法GDLLM。首先,利用图注意力网络(GAT)构建距离感知图结构,帮助大模型捕捉长距离依赖特征;其次,设计基于软推理的时序特征学习范式,增强短距离邻近关系的识别能力,并将大模型生成的概率信息融入多头注意力机制。该框架有效捕获全局特征,显著提升少数类关系的表现,整体学习能力也得到加强。在公开数据集TB-Dense和MATRES上的实验表明,本方法达到当前最优(SOTA)性能。
原文摘要 · Abstract (English)
In Natural Language Processing(NLP), Event Temporal Relation Extraction (ETRE) is to recognize the temporal relations of two events. Prior studies have noted the importance of language models for ETRE. However, the restricted pre-trained knowledge of Small Language Models(SLMs) limits their capability to handle minority class relations in imbalanced classification datasets. For Large Language Models(LLMs), researchers adopt manually designed prompts or instructions, which may introduce extra noise, leading to interference with the model's judgment of the long-distance dependencies between events. To address these issues, we propose GDLLM, a Global Distance-aware modeling approach based on LLMs. We first present a distance-aware graph structure utilizing Graph Attention Network(GAT) to assist the LLMs in capturing long-distance dependency features. Additionally, we design a temporal feature learning paradigm based on soft inference to augment the identification of relations with a short-distance proximity band, which supplements the probabilistic information generated by LLMs into the multi-head attention mechanism. Since the global feature can be captured effectively, our framework substantially enhances the performance of minority relation classes and improves the overall learning ability. Experiments on two publicly available datasets, TB-Dense and MATRES, demonstrate that our approach achieves state-of-the-art (SOTA) performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。