用大模型同时保留知识图谱结构和文本信息,提升推理准确率
Language Models as Ontology Encoders
- 用超球面几何优化预训练语言模型,融合文本标签与逻辑结构
- 在4个真实数据集上超越现有方法,预测和推理任务均表现更优
- 适合需要结构化知识建模的医疗、生物等领域的研究人员
OWL本体能形式化表示复杂知识并支持语义推理,已被广泛应用于医疗和生物信息学等领域。近年来,本体嵌入因具备推断新知识和近似复杂推理的潜力而受到关注。然而,现有方法存在明显局限:基于几何模型的嵌入通常忽略宝贵的文本信息,导致性能不佳;而依赖语言模型的方法虽引入文本,却难以保持逻辑结构。本文提出新方法OnT,通过在超球面空间中对预训练语言模型进行几何建模,有效结合文本标签并同时保留描述逻辑EL中的类层次及其他逻辑关系。在四个真实世界本体上的大量实验表明,OnT在命题预测与推理任务上持续优于基线模型,包括当前最优方法。OnT在实际应用中也展现出强大潜力,表现为出色的迁移学习能力,并成功用于从SNOMED CT构建新本体。代码与数据已公开于https://github.com/HuiYang1997/OnT。
原文摘要 · Abstract (English)
OWL (Web Ontology Language) ontologies which are able to formally represent complex knowledge and support semantic reasoning have been widely adopted across various domains such as healthcare and bioinformatics. Recently, ontology embeddings have gained wide attention due to its potential to infer plausible new knowledge and approximate complex reasoning. However, existing methods face notable limitations: geometric model-based embeddings typically overlook valuable textual information, resulting in suboptimal performance, while the approaches that incorporate text, which are often based on language models, fail to preserve the logical structure. In this work, we propose a new ontology embedding method OnT, which tunes a Pretrained Language Model (PLM) via geometric modeling in a hyperbolic space for effectively incorporating textual labels and simultaneously preserving class hierarchies and other logical relationships of Description Logic EL. Extensive experiments on four real-world ontologies show that OnT consistently outperforms the baselines including the state-of-the-art across both tasks of prediction and inference of axioms. OnT also demonstrates strong potential in real-world applications, indicated by its robust transfer learning abilities and effectiveness in real cases of constructing a new ontology from SNOMED CT. Data and code are available at https://github.com/HuiYang1997/OnT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。