用大模型自动生成知识图谱初稿,提升开发效率。
Ontology Generation using Large Language Models
- 提出两种新提示法,直接从用户需求生成本体
- o1-preview模型结合Ontogenia达专家级建模水平
- 适合需快速构建本体的开发者或研究者
本体工程过程复杂、耗时且易出错,即使对经验丰富的工程师也是如此。本文研究大语言模型(LLMs)从用户故事和能力问题中直接生成符合OWL规范的本体草案的潜力。主要贡献包括提出并评估两种新的自动化本体生成提示技术:Memoryless CQbyCQ 和 Ontogenia。我们强调三个结构标准在本体评估中的重要性,并结合专家定性评估,强调多维度评价对捕捉生成本体的质量与可用性的必要性。实验基于包含10个本体、100个能力问题和29个用户故事的基准数据集,对比三种LLM在两种提示技术下的表现。结果表明,OpenAI o1-preview 搭配 Ontogenia 所生成的本体质量足以满足工程师需求,在建模能力上显著优于新手。但仍存在常见错误和结果质量波动,使用时需注意。本文讨论了这些局限,并提出了未来研究方向。
原文摘要 · Abstract (English)
The ontology engineering process is complex, time-consuming, and error-prone, even for experienced ontology engineers. In this work, we investigate the potential of Large Language Models (LLMs) to provide effective OWL ontology drafts directly from ontological requirements described using user stories and competency questions. Our main contribution is the presentation and evaluation of two new prompting techniques for automated ontology development: Memoryless CQbyCQ and Ontogenia. We also emphasize the importance of three structural criteria for ontology assessment, alongside expert qualitative evaluation, highlighting the need for a multi-dimensional evaluation in order to capture the quality and usability of the generated ontologies. Our experiments, conducted on a benchmark dataset of ten ontologies with 100 distinct CQs and 29 different user stories, compare the performance of three LLMs using the two prompting techniques. The results demonstrate improvements over the current state-of-the-art in LLM-supported ontology engineering. More specifically, the model OpenAI o1-preview with Ontogenia produces ontologies of sufficient quality to meet the requirements of ontology engineers, significantly outperforming novice ontology engineers in modelling ability. However, we still note some common mistakes and variability of result quality, which is important to take into account when using LLMs for ontology authoring support. We discuss these limitations and propose directions for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。