自动挑选决策相关概念，让强化学习更可解释且高效。
Selecting Decision-Relevant Concepts in Reinforcement Learning
- 基于状态抽象思想，自动筛选对决策关键的概念。
- 选出的概念集性能接近甚至超越人工设计，有理论性能保证。
- 适用于强化学习基准和真实医疗场景，支持测试时干预。
训练可解释的概念驱动策略需要人工选择智能体在序列决策中应推理的人类可理解概念。这一过程依赖领域知识，耗时且成本高，随候选概念增多而难以扩展，且无性能保障。为此，我们提出首个面向序列决策的原理性自动概念选择算法。核心思路是将概念选择视为状态抽象问题：若移除某个概念会导致需采取不同动作的状态被混淆，则该概念即为决策相关。因此，智能体应依赖决策相关概念；具有相同概念表示的状态应共享最优动作，以保持原状态空间的最优决策结构。该视角导出决策相关选择（DRS）算法，从候选集自动选取概念子集，并提供所选概念与最终策略性能之间的理论边界。实验表明，DRS能自动恢复人工设计的概念集，性能相当或更优，并在强化学习基准及真实医疗环境中提升测试时概念干预的有效性。
原文摘要 · Abstract (English)
Training interpretable concept-based policies requires practitioners to manually select which human-understandable concepts an agent should reason with when making sequential decisions. This selection demands domain expertise, is time-consuming and costly, scales poorly with the number of candidates, and provides no performance guarantees. To overcome this limitation, we propose the first algorithms for principled automatic concept selection in sequential decision-making. Our key insight is that concept selection can be viewed through the lens of state abstraction: intuitively, a concept is decision-relevant if removing it would cause the agent to confuse states that require different actions. As a result, agents should rely on decision-relevant concepts; states with the same concept representation should share the same optimal action, which preserves the optimal decision structure of the original state space. This perspective leads to the Decision-Relevant Selection (DRS) algorithm, which selects a subset of concepts from a candidate set, along with performance bounds relating the selected concepts to the performance of the resulting policy. Empirically, DRS automatically recovers manually curated concept sets while matching or exceeding their performance, and improves the effectiveness of test-time concept interventions across reinforcement learning benchmarks and real-world healthcare environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。