提出快速近似强化学习的沙普利值解释方法
Approximating Shapley Explanations in Reinforcement Learning
- 基于合作博弈理论近似沙普利值，解决解释成本高问题
- 支持多步轨迹时序依赖与离策略数据学习
- 适合需要实时可解释性的安全关键场景
强化学习在复杂决策环境中取得了显著成功，但其缺乏透明性限制了实际应用，尤其在安全关键场景中。来自合作博弈论的沙普利值为强化学习提供了一个合理的解释框架，但计算沙普利解释的成本过高，成为实际应用的障碍。本文提出 FastSVERL，一种通过近似沙普利值来解释强化学习的可扩展方法。FastSVERL 针对强化学习的独特挑战进行了设计，包括多步轨迹中的时序依赖、从离策略数据中学习，以及适应实时演化的智能体行为。该方法提供了一种实用且可扩展的途径，实现强化学习中原则性且严谨的可解释性。
原文摘要 · Abstract (English)
Reinforcement learning has achieved remarkable success in complex decision-making environments, yet its lack of transparency limits its deployment in practice, especially in safety-critical settings. Shapley values from cooperative game theory provide a principled framework for explaining reinforcement learning; however, the computational cost of Shapley explanations is an obstacle to their use. We introduce FastSVERL, a scalable method for explaining reinforcement learning by approximating Shapley values. FastSVERL is designed to handle the unique challenges of reinforcement learning, including temporal dependencies across multi-step trajectories, learning from off-policy data, and adapting to evolving agent behaviours in real time. FastSVERL introduces a practical, scalable approach for principled and rigorous interpretability in reinforcement learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。