用支持向量机分块状态空间，让黑箱强化学习更易懂且高效。
Hierarchical Support Vector State Partitioning for Distilling Black Box Reinforcement Learning Policies
- 用线性SVM对状态动作数据分块，生成可解释的子策略。
- 相比VSP方法提升7.4%平均回报，子策略数量减少82.1%。
- 适合需要可解释性与低开销策略蒸馏的场景。
我们提出状态向量空间分割（SVSP），一种通过线性支持向量机对状态-动作对数据集进行划分的方法，以模仿黑箱强化学习策略并生成人类可理解的子策略。该方法构建了紧凑且结构化的原始策略表示，在平均回报上比基于评论家的先前状态分割方法如Voronoi状态分割（VSP）提升了7.4%，同时优于原始TD3策略2.8%。相较于VSP，SVSP将所需子策略数量减少了82.1%。结果表明，该方法为策略蒸馏提供了更灵活的路径，允许在原黑箱行为的容差范围内自由选择决策边界和代理模型。
原文摘要 · Abstract (English)
We introduce State Vector Space Partitioning (SVSP), a novel method to mimic a black box reinforcement learning policy using a set of human-interpretable subpolicies. By partitioning a distillation dataset of state action pairs with linear support vector machine splits, SVSP constructs a compact and structured representation of the original policy. Our method improves mean return by +7.4% over previous critic driven state partitioning attempts such as Voronoi State Partitioning (VSP) and +2.8% over the original TD3 policy, while reducing the number of required subpolicies against VSP by 82.1%. Our results pave the path towards a more flexible form of distillation where both the decision boundary and surrogate models can be chosen within a margin of the original black box behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。