arXiv:2410.23583cs.CLcs.LG2024-10

无需实体标注,用非对比学习提升生物文本关系抽取效果

BioNCERE: Non-Contrastive Enhancement For Relation Extraction In Biomedical Texts

  • 采用两次迁移学习与非对比学习,避免维度坍塌和过拟合
  • 在SemMedDB数据集上达到接近顶尖水平的性能
  • 无需实体标签,降低标注成本,适合资源受限场景

生物医学领域的关系抽取(RE)主流模型通常基于BioBERT微调进行分类,但易受方向性问题影响。对比学习可缓解此问题并防止类别坍塌。本文提出一种名为生物非对比关系抽取(BioNCERE)的新训练方法,不依赖命名实体标签进行训练,从而降低标注成本。该方法结合迁移学习与非对比学习,在三个阶段中分步完成关系抽取:前两阶段使用迁移学习,第二阶段冻结前期权重并引入非对比学习,使模型在不依赖实体知识的情况下预测关系。在SemMedDB数据集上的实验表明,其性能接近现有最先进水平,且无需实体信息。

原文摘要 · Abstract (English)

State-of-the-art models for relation extraction (RE) in the biomedical domain consider finetuning BioBERT using classification, but they may suffer from the anisotropy problem. Contrastive learning methods can reduce this anisotropy phenomena, and also help to avoid class collapse in any classification problem. In the present paper, a new training method called biological non-contrastive relation extraction (BioNCERE) is introduced for relation extraction without using any named entity labels for training to reduce annotation costs. BioNCERE uses transfer learning and non-contrastive learning to avoid full or dimensional collapse as well as bypass overfitting. It resolves RE in three stages by leveraging transfer learning two times. By freezing the weights learned in previous stages in the proposed pipeline and by leveraging non-contrastive learning in the second stage, the model predicts relations without any knowledge of named entities. Experiments have been done on SemMedDB that are almost similar to State-of-the-art performance on RE without using the information of named entities.

关系抽取非对比学习生物文本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。