用大模型自动生成可理解的问答系统决策解释。
Towards LLM-generated explanations for Component-based Knowledge Graph Question Answering Systems
- 以组件输入输出数据流为依据生成解释。
- 大模型生成的解释用户评分更高，优于模板方法。
- 适合需要理解复杂问答系统决策的人群。
随着软件系统复杂度提升，开发者和用户难以解释其决策过程。本文聚焦于基于组件的问答（QA）系统解释性问题，这些组件常依赖人工智能方法，其行为和决策难以清晰说明，连专家也难解读。为此，我们提出一种方法：将组件的输入输出数据流作为行为表示，用于生成解释。在所用的QA框架中，输入为SPARQL查询，输出为RDF三元组，因此我们也提供了针对这些数据类型的自然语言化见解。实验中，我们比较了基于模板的解释生成（基线）与使用不同配置的大语言模型（LLMs）进行自动生成的效果。评估结果显示，由大模型生成的解释质量更高，且多数情况下优于模板方法。该方法能利用RDF和SPARQL作为上下文，自动向人类解释问答组件的行为与决策。
原文摘要 · Abstract (English)
Over time, software systems have reached a level of complexity that makes it difficult for their developers and users to explain particular decisions made by them. In this paper, we focus on the explainability of component-based systems for Question Answering (QA). These components often conduct processes driven by AI methods, in which behavior and decisions cannot be clearly explained or justified, s.t., even for QA experts interpreting the executed process and its results is hard. To address this challenge, we present an approach that considers the components' input and output data flows as a source for representing the behavior and provide explanations for the components, enabling users to comprehend what happened. In the QA framework used here, the data flows of the components are represented as SPARQL queries (inputs) and RDF triples (outputs). Hence, we are also providing valuable insights on verbalization regarding these data types. In our experiments, the approach generates explanations while following template-based settings (baseline) or via the use of Large Language Models (LLMs) with different configurations (automatic generation). Our evaluation shows that the explanations generated via LLMs achieve high quality and mostly outperform template-based approaches according to the users' ratings. Therefore, it enables us to automatically explain the behavior and decisions of QA components to humans while using RDF and SPARQL as a context for explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。