用可解释的特征权重改进强化学习，让模型行为更透明
VRAIL: Vectorized Reward-based Attribution for Interpretable Learning
- 分两阶段学习：先拟合价值函数，再用其改造奖励信号
- 在Taxi-v3上训练更稳定，收敛更快，无需修改环境
- 能识别出乘客携带等有意义子目标，适合需要可解释性的场景
我们提出VRAIL（基于向量奖励的可解释学习归因），一种用于基于价值的强化学习的双层框架，从状态特征中学习可解释的权重表示。VRAIL包含两个阶段：第一阶段使用状态特征拟合估计的价值函数，第二阶段利用该函数通过基于势能的奖励变换来引导学习。估值器采用线性或二次形式建模，可对单个特征及其交互作用进行重要性归因。在Taxi-v3环境上的实证结果表明，与标准DQN相比，VRAIL提升了训练稳定性与收敛性，且无需修改环境。进一步分析显示，VRAIL揭示了如乘客持有等语义上合理的子目标，凸显其生成人类可理解行为的能力。研究结果表明，VRAIL是一种通用、模型无关的奖励塑形框架，同时提升学习效果与可解释性。
原文摘要 · Abstract (English)
We propose VRAIL (Vectorized Reward-based Attribution for Interpretable Learning), a bi-level framework for value-based reinforcement learning (RL) that learns interpretable weight representations from state features. VRAIL consists of two stages: a deep learning (DL) stage that fits an estimated value function using state features, and an RL stage that uses this to shape learning via potential-based reward transformations. The estimator is modeled in either linear or quadratic form, allowing attribution of importance to individual features and their interactions. Empirical results on the Taxi-v3 environment demonstrate that VRAIL improves training stability and convergence compared to standard DQN, without requiring environment modifications. Further analysis shows that VRAIL uncovers semantically meaningful subgoals, such as passenger possession, highlighting its ability to produce human-interpretable behavior. Our findings suggest that VRAIL serves as a general, model-agnostic framework for reward shaping that enhances both learning and interpretability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。