arXiv:2512.05594cs.AIcs.CL2025-12被引 3

用大模型自动识别知识图谱中的逻辑规则,提升领域知识构建效率

Ontology Learning with LLMs: A Benchmark Study on Axiom Identification

  • 构建了包含2771条公理的基准测试集,评估大模型在五类逻辑关系上的识别能力
  • 分步提问比一次性提问更准确,但不同类别公理难易差异明显
  • 大模型可生成候选公理,辅助工程师快速构建知识库

本研究聚焦于知识图谱构建中的核心挑战:识别公理——定义类与属性间逻辑关系的基本单元。我们提出了首个系统性基准测试集OntoAxiom,涵盖九个中等规模的本体,共含17,118个三元组和2,771条公理,覆盖子类、互斥、子属性、域、范围五类关键关系。通过对比十二个大模型在两种提示策略下的表现(一次性提问与逐条提问),发现逐条提问(AbA)能显著提升F1分数。然而,性能受领域影响显著:如FOAF本体的子类公理得分达0.642,而音乐本体仅0.218。大模型整体表现尚不足以完全自动化公理识别,但可为本体工程师提供高质量候选,有效支持知识建模与优化。

原文摘要 · Abstract (English)

Ontologies are an important tool for structuring domain knowledge, but their development is a complex task that requires significant modelling and domain expertise. Ontology learning, aimed at automating this process, has seen advancements in the past decade with the improvement of Natural Language Processing techniques, and especially with the recent growth of Large Language Models (LLMs). This paper investigates the challenge of identifying axioms: fundamental ontology components that define logical relations between classes and properties. In this work, we introduce an Ontology Axiom Benchmark OntoAxiom, and systematically test LLMs on that benchmark for axiom identification, evaluating different prompting strategies, ontologies, and axiom types. The benchmark consists of nine medium-sized ontologies with together 17.118 triples, and 2.771 axioms. We focus on subclass, disjoint, subproperty, domain, and range axioms. To evaluate LLM performance, we compare twelve LLMs with three shot settings and two prompting strategies: a Direct approach where we query all axioms at once, versus an Axiom-by-Axiom (AbA) approach, where each prompt queries for one axiom only. Our findings show that the AbA prompting leads to higher F1 scores than the direct approach. However, performance varies across axioms, suggesting that certain axioms are more challenging to identify. The domain also influences performance: the FOAF ontology achieves a score of 0.642 for the subclass axiom, while the music ontology reaches only 0.218. Larger LLMs outperform smaller ones, but smaller models may still be viable for resource-constrained settings. Although performance overall is not high enough to fully automate axiom identification, LLMs can provide valuable candidate axioms to support ontology engineers with the development and refinement of ontologies.

知识图谱大模型本体学习自动推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。