用外部知识增强语言模型,提升生物医学关系抽取效果
Knowledge-augmented Pre-trained Language Models for Biomedical Relation Extraction
- 在统一框架下测试多种模型与参数优化策略
- 小模型受益于外部知识,大模型提升有限
- 适合关注生物医学文本挖掘的研究者
从生物医学文献中自动提取关系对管理海量科学知识至关重要。近年来,预训练语言模型(PLMs)已成为关系抽取的主流方法。一些研究发现,在微调时引入额外上下文信息可提升性能。然而,不同模型、知识库、超参数和评估方式使研究间难以直接比较,也引发结果泛化性的质疑。本研究通过统一评估框架,在五个数据集上覆盖四种关系场景,评估三种基线模型,并进行广泛超参数优化。选定最优模型后,引入文本实体描述、知识图谱中的关系信息及分子结构编码进行增强。结果表明:1)底层语言模型的选择至关重要;2)全面的超参数优化对性能影响显著。尽管整体上增加上下文信息带来的提升有限,消融实验显示,小模型在微调时加入外部数据能获得显著收益。
原文摘要 · Abstract (English)
Automatic relationship extraction (RE) from biomedical literature is critical for managing the vast amount of scientific knowledge produced each year. In recent years, utilizing pre-trained language models (PLMs) has become the prevalent approach in RE. Several studies report improved performance when incorporating additional context information while fine-tuning PLMs for RE. However, variations in the PLMs applied, the databases used for augmentation, hyper-parameter optimization, and evaluation methods complicate direct comparisons between studies and raise questions about the generalizability of these findings. Our study addresses this research gap by evaluating PLMs enhanced with contextual information on five datasets spanning four relation scenarios within a consistent evaluation framework. We evaluate three baseline PLMs and first conduct extensive hyperparameter optimization. After selecting the top-performing model, we enhance it with additional data, including textual entity descriptions, relational information from knowledge graphs, and molecular structure encodings. Our findings illustrate the importance of i) the choice of the underlying language model and ii) a comprehensive hyperparameter optimization for achieving strong extraction performance. Although inclusion of context information yield only minor overall improvements, an ablation study reveals substantial benefits for smaller PLMs when such external data was included during fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。