用大模型生成可验证的专家系统知识，兼顾准确与透明
GOFAI meets Generative AI: Development of Expert Systems by means of Large Language Models
- 限定领域+结构化提示提取，将LLM输出转为可验证的Prolog知识
- 在Claude Sonnet 3.7和GPT-4.1上实现高事实一致性与语义连贯性
- 适合医疗、金融等需可信推理的敏感领域应用
大语言模型（LLMs）已成功推动开放域问答等基于知识的系统发展，能自动生成大量看似连贯的信息。然而，这些模型存在幻觉或自信生成错误或不可验证事实的问题。本文提出一种使用LLMs可控且透明地开发专家系统的新方法：通过限定领域并采用结构化提示提取，生成可由人类专家验证与修正的Prolog符号知识表示。该方法保证了系统的可解释性、可扩展性与可靠性。通过对Claude Sonnet 3.7与GPT-4.1进行定量与定性实验，验证了生成知识库在事实准确性与语义连贯性上的强表现。本文提出了一种透明的混合方案，结合了LLMs的召回能力与符号系统的精确性，为敏感领域的可靠AI应用奠定基础。
原文摘要 · Abstract (English)
The development of large language models (LLMs) has successfully transformed knowledge-based systems such as open domain question nswering, which can automatically produce vast amounts of seemingly coherent information. Yet, those models have several disadvantages like hallucinations or confident generation of incorrect or unverifiable facts. In this paper, we introduce a new approach to the development of expert systems using LLMs in a controlled and transparent way. By limiting the domain and employing a well-structured prompt-based extraction approach, we produce a symbolic representation of knowledge in Prolog, which can be validated and corrected by human experts. This approach also guarantees interpretability, scalability and reliability of the developed expert systems. Via quantitative and qualitative experiments with Claude Sonnet 3.7 and GPT-4.1, we show strong adherence to facts and semantic coherence on our generated knowledge bases. We present a transparent hybrid solution that combines the recall capacity of LLMs with the precision of symbolic systems, thereby laying the foundation for dependable AI applications in sensitive domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。