将复杂强化学习策略拆解为局部可解释的线性模型，提升透明度与可信度。
Explainable RL Policies by Distilling to Locally-Specialized Linear Policies with Voronoi State Partitioning
- 用Voronoi分割状态空间，每个区域部署一个简单线性模型。
- 在网格世界和经典控制任务中，解释型策略性能媲美甚至优于原黑箱模型。
- 适合需可信决策的场景，如自动驾驶、医疗控制等安全敏感领域。
深度强化学习是生成近最优系统控制器的前沿方法，但其训练的深层神经网络缺乏透明性，难以满足监管要求或建立信任。为缓解此问题，可通过知识蒸馏将学习行为迁移到设计上可读的人类理解模型。传统方法通常使用单一模型平均模仿原模型，但在动态环境中表现不佳。关键挑战在于平衡简化模型的灵活性与复杂性，避免偏差与精度失衡。本文提出一种模型无关的方法，通过Voronoi状态空间划分，将环境分成若干区域，每个区域内部署一个简化且可解释的线性模型。实验在网格世界和经典控制任务上进行，结果表明：所提的局部专用线性模型蒸馏方法生成的策略具备可解释性，且性能匹配甚至略微超越原黑箱策略。
原文摘要 · Abstract (English)
Deep Reinforcement Learning is one of the state-of-the-art methods for producing near-optimal system controllers. However, deep RL algorithms train a deep neural network, that lacks transparency, which poses challenges when the controller has to meet regulations, or foster trust. To alleviate this, one could transfer the learned behaviour into a model that is human-readable by design using knowledge distilla- tion. Often this is done with a single model which mimics the original model on average but could struggle in more dynamic situations. A key challenge is that this simpler model should have the right balance be- tween flexibility and complexity or right balance between balance bias and accuracy. We propose a new model-agnostic method to divide the state space into regions where a simplified, human-understandable model can operate in. In this paper, we use Voronoi partitioning to find regions where linear models can achieve similar performance to the original con- troller. We evaluate our approach on a gridworld environment and a classic control task. We observe that our proposed distillation to locally- specialized linear models produces policies that are explainable and show that the distillation matches or even slightly outperforms the black-box policy they are distilled from.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。