arXiv:2411.16120cs.AIcs.LG2024-11IJCAI被引 2

让强化学习决策可解释,回答‘为何选这个而非那个’

Why the Agent Made that Decision: Contrastive Explanation Learning for Reinforcement Learning

  • 通过对比学习,让模型解释为何选当前动作而非其他选项
  • 在多个环境测试中提升解释的忠实性与鲁棒性,保持决策准确
  • 适合需要理解智能体行为的高风险场景,如医疗或自动驾驶

强化学习在复杂决策问题上表现卓越,但在关键领域应用受限于其决策过程缺乏可解释性。现有可解释AI方法常无法提供有意义的解释,尤其忽视了人类推理中的对比本质——回答‘为何选这个而非那个’。为此,我们提出一种新的对比解释学习框架VisionMask,通过自监督方式显式对比智能体在特定状态下的选择动作与其他备选动作,生成解释。我们在多种强化学习环境中验证该方法的有效性,评估其忠实性、鲁棒性和复杂度。结果表明,VisionMask显著提升了人类对智能体行为的理解,同时保持高准确率与高保真度。此外,我们展示了该方法在反事实分析中的应用实例。本工作弥合了强化学习与可解释AI之间的鸿沟,为更安全、可解释的强化学习系统铺平道路。

原文摘要 · Abstract (English)

Reinforcement learning (RL) has demonstrated remarkable success in solving complex decision-making problems, yet its adoption in critical domains is hindered by the lack of interpretability in its decision-making processes. Existing explainable AI (xAI) approaches often fail to provide meaningful explanations for RL agents, particularly because they overlook the contrastive nature of human reasoning--answering "why this action instead of that one?". To address this gap, we propose a novel framework of contrastive learning to explain RL selected actions, named $\textbf{VisionMask}$. VisionMask is trained to generate explanations by explicitly contrasting the agent's chosen action with alternative actions in a given state using a self-supervised manner. We demonstrate the efficacy of our method through experiments across diverse RL environments, evaluating it in terms of faithfulness, robustness, and complexity. Our results show that VisionMask significantly improves human understanding of agent behavior while maintaining accuracy and fidelity. Furthermore, we present examples illustrating how VisionMask can be used for counterfactual analysis. This work bridges the gap between RL and xAI, paving the way for safer and more interpretable RL systems.

强化学习可解释AI对比学习决策解释

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。