arXiv:2511.07991cs.AI2025-11AAAI被引 2

用大模型生成能检测本体语义错误的问答,提升自动验证效率。

VSPO: Validating Semantic Pitfalls in Ontology via LLM-Based CQ Generation

  • 通过修改本体公理并生成不匹配的定义,模拟语义错误。
  • 模型在检测错误上比GPT-4.1高26%精确率和28.2%召回率。
  • 适合本体工程、知识图谱构建者用于自动化质量验证。

能力问题(CQs)在本体设计验证中至关重要。传统手动构建耗时费力,现有基于大语言模型(LLMs)的方法多依赖与已有数据集的相似性评估,难以发现如"Misusing allValuesFrom"等语义缺陷。为此,我们提出首个专为检测语义陷阱设计的VSPO数据集与模型。通过LLM生成类与属性的自然语言定义,并通过移除公理或替换逻辑算子(如将并集改为交集)制造定义与本体间的不一致。随后微调LLaMA-3.1-8B-Instruct模型生成可验证这些差异的CQs。结果表明,该模型生成的CQs能检测更广泛的建模错误,相较基线模型,精度提升26%,召回率提高28.2%。本研究首次实现基于大模型自动生成用于TBox验证的语义陷阱检测型CQs,显著降低人工成本,增强本体与专家知识的语义对齐。

原文摘要 · Abstract (English)

Competency Questions (CQs) play a crucial role in validating ontology design. While manually crafting CQs can be highly time-consuming and costly for ontology engineers, recent studies have explored the use of large language models (LLMs) to automate this process. However, prior approaches have largely evaluated generated CQs based on their similarity to existing datasets, which often fail to verify semantic pitfalls such as "Misusing allValuesFrom". Since such pitfalls cannot be reliably detected through rule-based methods, we propose a novel dataset and model of Validating Semantic Pitfalls in Ontology (VSPO) for CQ generation specifically designed to verify the semantic pitfalls. To simulate missing and misused axioms, we use LLMs to generate natural language definitions of classes and properties and introduce misalignments between the definitions and the ontology by removing axioms or altering logical operators (e.g., substituting union with intersection). We then fine-tune LLaMA-3.1-8B-Instruct to generate CQs that validate these semantic discrepancies between the provided definitions and the corresponding axioms. The resulting CQs can detect a broader range of modeling errors compared to existing public datasets. Our fine-tuned model demonstrates superior performance over baselines, showing 26% higher precision and 28.2% higher recall than GPT-4.1 in generating CQs for pitfall validation. This research enables automatic generation of TBox-validating CQs using LLMs, significantly reducing manual effort while improving semantic alignment between ontologies and expert knowledge. To the best of our knowledge, this is the first study to target semantic pitfall validation in CQ generation using LLMs.

本体验证大模型应用语义错误检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。