用强化学习让小模型学会按规范推理，提升跨领域关系抽取能力
R1-RE: Cross-Domain Relation Extraction with RLVR
- 将关系抽取重构为受标注规范引导的推理任务，采用可验证奖励机制训练
- 70%跨域准确率，接近GPT-4o水平，显著优于传统监督方法
- 适合关注小模型推理能力、跨领域泛化与可解释性研究的读者
关系抽取（RE）是自然语言处理的核心任务。传统方法通常将其视为监督学习问题，直接从上下文映射到标签，常导致跨领域泛化能力差。受人工标注流程启发，我们重新将RE视为受标注指南引导的推理任务，提出首个基于可验证奖励的强化学习框架（RLVR）——R1-RE。该方法激发小型语言模型在标注任务中的推理能力，显著提升跨领域鲁棒性。我们在公开的Sem-2010数据集和一个私有MDKG数据集上评估，R1-RE-7B模型在跨域任务中平均准确率达约70%，媲美领先专有模型GPT-4o。此外，我们的全面分析揭示了RLVR范式在RE任务中的训练动态与涌现推理行为。
原文摘要 · Abstract (English)
Relation extraction (RE) is a core task in natural language processing. Traditional approaches typically frame RE as a supervised learning problem, directly mapping context to labels-an approach that often suffers from poor out-of-domain (OOD) generalization. Inspired by the workflow of human annotators, we reframe RE as a reasoning task guided by annotation guidelines and introduce R1-RE, the first reinforcement learning with verifiable reward (RLVR) framework for RE tasks. Our method elicits the reasoning abilities of small language models for annotation tasks, resulting in significantly improved OOD robustness. We evaluate our approach on the public Sem-2010 dataset and a private MDKG dataset. The R1-RE-7B model attains an average OOD accuracy of approximately 70%, on par with leading proprietary models such as GPT-4o. Additionally, our comprehensive analysis provides novel insights into the training dynamics and emergent reasoning behaviors of the RLVR paradigm for RE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。