用生成式与判别式结合方法,解决少样本关系抽取数据不足问题
Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training
- 利用大模型生成带解释的合成数据和符合模式的伪标签
- 两阶段预训练提升关系推理能力,少样本下准确率超现有方法
- 适合低资源场景的关系抽取研究者使用
少样本关系抽取(FSRE)因标注数据稀缺和模型泛化能力弱而面临挑战。尽管大语言模型(LLMs)通过上下文学习(ICL)展现出潜力,但其通用训练目标导致任务特定性能不佳。为此,我们提出TKRE框架,融合生成式与判别式学习,实现两类模型协同。该方法创新性地:(1)利用LLM生成解释驱动的知识与模式约束的合成数据,缓解数据稀缺;(2)采用两阶段预训练策略,结合掩码跨度语言建模(MSLM)与跨度级对比学习(SCL),增强关系推理与泛化能力。在多个基准数据集上的实验表明,TKRE达到新的最优性能,验证了其在低资源场景下的广泛应用潜力。
原文摘要 · Abstract (English)
Few-Shot Relation Extraction (FSRE) remains a challenging task due to the scarcity of annotated data and the limited generalization capabilities of existing models. Although large language models (LLMs) have demonstrated potential in FSRE through in-context learning (ICL), their general-purpose training objectives often result in suboptimal performance for task-specific relation extraction. To overcome these challenges, we propose TKRE (Two-Stage Knowledge-Guided Pre-training for Relation Extraction), a novel framework that synergistically integrates LLMs with traditional relation extraction models, bridging generative and discriminative learning paradigms. TKRE introduces two key innovations: (1) leveraging LLMs to generate explanation-driven knowledge and schema-constrained synthetic data, addressing the issue of data scarcity; and (2) a two-stage pre-training strategy combining Masked Span Language Modeling (MSLM) and Span-Level Contrastive Learning (SCL) to enhance relational reasoning and generalization. Together, these components enable TKRE to effectively tackle FSRE tasks. Comprehensive experiments on benchmark datasets demonstrate the efficacy of TKRE, achieving new state-of-the-art performance in FSRE and underscoring its potential for broader application in low-resource scenarios. \footnote{The code and data are released on https://github.com/UESTC-GQJ/TKRE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。