让复杂智能体的意图可解释，回答‘想做什么、如何做、为何这么做’。
Intention-aware policy graphs: answering what, how, and why in opaque agents
- 用概率图模型建模智能体行为，推断其即时意图。
- 能计算意图可信度，并支持对行为合理性评估。
- 适合需要理解黑箱智能体决策的开发者与监管者。
智能体是能在复杂环境中交互的AI软件，其潜在的涌现行为使可信AI部署面临挑战。本文提出一种概率图模型及构建流程，用于推断智能体在任意时刻的意图，并提供可量化评估的解释性指标。该模型能回答‘现在想做什么’（如送汤）、‘如何实现’（如结合自身技能与环境状态制定计划）以及‘为何采取此行动’（如说明该行为如何促进或阻碍目标）。模型基于对智能体动作和世界状态的部分观测构建，支持通过迭代优化提升解释质量，或揭示智能体的不合理行为。研究还提供了评估解释可读性和可靠性的方法。
原文摘要 · Abstract (English)
Agents are a special kind of AI-based software in that they interact in complex environments and have increased potential for emergent behaviour. Explaining such emergent behaviour is key to deploying trustworthy AI, but the increasing complexity and opaque nature of many agent implementations makes this hard. In this work, we propose a Probabilistic Graphical Model along with a pipeline for designing such model -- by which the behaviour of an agent can be deliberated about -- and for computing a robust numerical value for the intentions the agent has at any moment. We contribute measurements that evaluate the interpretability and reliability of explanations provided, and enables explainability questions such as `what do you want to do now?' (e.g. deliver soup) `how do you plan to do it?' (e.g. returning a plan that considers its skills and the world), and `why would you take this action at this state?' (e.g. explaining how that furthers or hinders its own goals). This model can be constructed by taking partial observations of the agent's actions and world states, and we provide an iterative workflow for increasing the proposed measurements through better design and/or pointing out irrational agent behaviour.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。