arXiv:2409.05275cs.CL2024-09被引 1

统一信息抽取与文本分类,用显式模式约束提升模型泛化能力

RexUniNLU: Recursive Method with Explicit Schema Instructor for Universal NLU

  • 提出递归方法+显式模式引导,统一处理各类信息抽取与分类任务
  • 在中英文及多模态数据上表现优于现有模型,准确率提升显著
  • 适合需要通用自然语言理解的场景,如跨任务系统集成

信息抽取(IE)和文本分类(CLS)是自然语言理解的基础任务,均依赖对输入序列的分析以将输出归类到预定义模式中。然而,目前尚无基于编码器的模型能从这一视角统一处理这两类任务。为充分挖掘自然语言理解任务间的共性基础,本文提出递归方法与显式模式引导的通用自然语言理解框架(RexUniNLU)。首先,我们形式化定义了涵盖几乎所有抽取模式(包括四元组、五元组等此前未解决类型)的通用信息抽取(UIE)问题;随后,将该形式化扩展至所有文本分类及多模态自然语言理解任务。基于此,构建RexUniNLU,通过显式模式约束实现对IE与CLS任务的统一建模,避免模式与输入序列之间的错误关联。为减少不同模式间的干扰,重新设置位置编码与注意力掩码矩阵。在中英文文本分类、信息抽取及多模态任务上的大量实验表明其有效性与优越性。代码已开源。

原文摘要 · Abstract (English)

Information Extraction (IE) and Text Classification (CLS) serve as the fundamental pillars of NLU, with both disciplines relying on analyzing input sequences to categorize outputs into pre-established schemas. However, there is no existing encoder-based model that can unify IE and CLS tasks from this perspective. To fully explore the foundation shared within NLU tasks, we have proposed a Recursive Method with Explicit Schema Instructor for Universal NLU. Specifically, we firstly redefine the true universal information extraction (UIE) with a formal formulation that covers almost all extraction schemas, including quadruples and quintuples which remain unsolved for previous UIE models. Then, we expands the formulation to all CLS and multi-modal NLU tasks. Based on that, we introduce RexUniNLU, an universal NLU solution that employs explicit schema constraints for IE and CLS, which encompasses all IE and CLS tasks and prevent incorrect connections between schema and input sequence. To avoid interference between different schemas, we reset the position ids and attention mask matrices. Extensive experiments are conducted on IE, CLS in both English and Chinese, and multi-modality, revealing the effectiveness and superiority. Our codes are publicly released.

通用NLU信息抽取文本分类显式约束

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。