用大模型从文本中提取操作步骤，构建可执行的知识图谱。
Human Evaluation of Procedural Knowledge Graph Extraction from Text with Large Language Models
- 设计提示工程方法，从文本中抽取步骤、动作、对象等信息
- 用户评估显示大模型输出质量可接受，能有效支持知识结构化
- 适合需要自动化处理手册、流程文档的工业或医疗场景
程序性知识是描述完成任务所需步骤序列的实践性知识，常见于食谱、维护手册等自然语言文本中，常分散在不同文档或系统中，其理解与执行依赖读者自行判断。将此类知识以知识图谱（KG）形式表示，可为用户提供数字化辅助工具。本文利用大语言模型（LLM）能力，提出一种提示工程方法，从文本中提取步骤、动作、对象、设备及时间信息，并依据预定义本体构建程序性知识图谱。通过用户研究对提取结果进行定性与定量评估，分析人类对AI生成知识的质量感知与实用性认知。结果表明，LLM能够生成可接受质量的输出，且用户对基于AI的知识表示持积极态度。
原文摘要 · Abstract (English)
Procedural Knowledge is the know-how expressed in the form of sequences of steps needed to perform some tasks. Procedures are usually described by means of natural language texts, such as recipes or maintenance manuals, possibly spread across different documents and systems, and their interpretation and subsequent execution is often left to the reader. Representing such procedures in a Knowledge Graph (KG) can be the basis to build digital tools to support those users who need to apply or execute them. In this paper, we leverage Large Language Model (LLM) capabilities and propose a prompt engineering approach to extract steps, actions, objects, equipment and temporal information from a textual procedure, in order to populate a Procedural KG according to a pre-defined ontology. We evaluate the KG extraction results by means of a user study, in order to qualitatively and quantitatively assess the perceived quality and usefulness of the LLM-extracted procedural knowledge. We show that LLMs can produce outputs of acceptable quality and we assess the subjective perception of AI by human evaluators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。