为遵守规则的智能体设计可解释框架，通过反事实分析说明行为原因。
Explainability Framework for Policy-Aware Autonomous Agents
- 基于社会学原理构建可理解解释框架，结合程序逻辑与自然语言生成。
- 利用违规惩罚机制识别反事实场景下的不良事件，生成对比性解释。
- 面向需要透明决策过程的高风险应用，如自动驾驶或医疗系统。
在人工智能领域，智能体是能够自主决策以达成目标的系统。随着这类系统日益融入日常生活，对其行为提供可解释性变得愈发重要。本文提出一个框架，用于为具有规则约束政策的智能体生成可理解的解释。该框架借鉴社会科学研究中良好解释的标准，采用答案集编程实现，并用Python辅助信息抽取与自然语言转换。由于违反政策会触发惩罚机制，我们可借此检测与智能体原始行为相反情境下的不良事件，从而生成对比性解释（例如：“该智能体执行此操作，是因为若不如此，将导致不良事件X”），构成解释框架的核心。通过问卷调查评估了由程序生成的解释，参与者提供了反馈意见。
原文摘要 · Abstract (English)
In the field of Artificial Intelligence, an agent is a system which is able to autonomously make decisions in order to reach a desired goal. As these systems grow more prevalent in our day-to-day lives, there has been an increased need to add explainability features which can provide an account for an agent's behavior. We therefore propose a framework that outlines how to produce comprehensible explanations for policy-aware agents, or agents which have rule-enforcing policies incorporated in their decision-making framework. This framework is designed using insights from the social sciences on how to produce good explanations. It is implemented in the Answer Set Programming language while using Python to assist with information extraction and natural-language translation. Because these agents incur penalties when violating policies, we are able to leverage these penalties to detect undesirable events in scenarios that are counterfactual to the agents' original actions. This lends itself to creating contrastive explanations (e.g., "the agent performed this action because, had it not, undesirable event X would have occurred."), which formulate the core component for our explainability framework. The framework is evaluated using a survey wherein human participants provide feedback on our program-generated explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。