研究自然语言查询为何难验证，提出用大模型辅助优化查询以减少建模错误。
When CQs Go Wrong: Challenges in CQ Verification with OE-Assist
- 用大模型助手协助用户完成20项本体验证任务
- 19名参与者发现模糊或复杂的查询导致验证结果不一致
- 建议发布前用工具优化查询，避免后期建模出错
能力问题（CQs）是CQ-验证的核心，该过程通过自然语言问题评估本体是否正确建模其预期用途。然而，该过程常耗时且易出错，需精确理解语言细节并匹配形式化本体结构。查询中的模糊性和复杂性会加剧问题，导致建模决策和验证结果不一致。本文研究使CQ难以验证的原因及改进方案，实验基于19名参与者在20个任务上使用大模型助手进行本体评估的数据。结果显示，在发布前用工具优化查询，可有效避免后续阶段的歧义或过度复杂化。
原文摘要 · Abstract (English)
Competency Questions (CQs) are the central component of CQ-verification, an established process in which an ontology is evaluated against a set of natural language questions to determine whether the intended purpose of the ontology has been properly modelled. However, CQ-verification is often time-consuming and error-prone, as it requires careful interpretation of linguistic nuances and precise alignment with formal ontology constructs. Ambiguities and complexity in CQs can further complicate this process, leading to inconsistent modelling decisions and verification outcomes. In this paper, we investigate what makes a CQ challenging and possible solutions to enhance the users' performance in the CQ-verification process. We experimented with the data of 19 participants who performed CQ-verification on 20 tasks using an LLM assistant to support ontology evaluation. The results show the necessity of a tool to refine CQs before publishing them to avoid ambiguity or excessive complexity in later phases of the ontology engineering process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。