用可观察行为评估解释效果，让XRL研究更可靠。
Objective Metrics for Human-Subjects Evaluation in Explainable Reinforcement Learning
- 以用户实际行为代替主观感受作为评估依据
- 在新网格环境中验证了多种客观评价方法
- 适合关注可复现性与实证效果的研究者
解释是根本上的人类过程。理解解释的目标与受众至关重要，但现有的可解释强化学习（XRL）研究通常未在评估中咨询人类。即使涉及人类，也常依赖信心或理解度等主观指标，仅反映用户意见，无法衡量解释在具体问题中的实际效用。本文呼吁研究者采用基于可观测、可行动行为的客观人类指标来评估解释效果，以建立更具可重复性、可比性和认识论基础的研究体系。为此，我们整理、描述并对比了几种客观评估方法，应用于调试智能体行为和辅助人机协作，并通过一个新型网格环境展示了这些方法的应用。文章还讨论了主观与客观指标如何互补以实现全面验证，强调未来工作需使用标准化基准测试，以促进研究间的有效比较。
原文摘要 · Abstract (English)
Explanation is a fundamentally human process. Understanding the goal and audience of the explanation is vital, yet existing work on explainable reinforcement learning (XRL) routinely does not consult humans in their evaluations. Even when they do, they routinely resort to subjective metrics, such as confidence or understanding, that can only inform researchers of users' opinions, not their practical effectiveness for a given problem. This paper calls on researchers to use objective human metrics for explanation evaluations based on observable and actionable behaviour to build more reproducible, comparable, and epistemically grounded research. To this end, we curate, describe, and compare several objective evaluation methodologies for applying explanations to debugging agent behaviour and supporting human-agent teaming, illustrating our proposed methods using a novel grid-based environment. We discuss how subjective and objective metrics complement each other to provide holistic validation and how future work needs to utilise standardised benchmarks for testing to enable greater comparisons between research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。