让强化学习更透明：用物理知识指导的树模型替代黑箱策略
Explainable Reinforcement Learning via Physics-Aware Policy Distillation
- 用物理感知特征和噪声模拟数据，将复杂策略蒸馏为简单决策树
- 蒸馏后模型性能与原始黑箱模型相当，且保持系统稳定
- 适合需要可解释性的机器人、自动驾驶等高安全场景
在机器人、汽车工程等高安全性领域，深度强化学习因神经网络的黑箱特性而难以部署。本文通过实验研究提升连续控制DRL系统的可解释性。采用经典倒立摆基准测试，以高性能的连续型TD3智能体作为教师模型，将其策略蒸馏至基于浅层决策树的学生模型。利用自定义的物理感知特征与“噪声模拟回放”生成训练数据，蒸馏后模型性能与专家教师相当。对比控制理论分析表明，从连续控制转为离散规则控制会引发高频开关动作及稳定的双模极限环。仿真结果表明，系统在保持有界输入有界输出（BIBO）稳定性的同时，实现了全局与局部双重可解释性，适用于安全自主系统。
原文摘要 · Abstract (English)
In safety-critical sectors such as robotics and automotive engineering, the deployment of Deep Reinforcement Learning (DRL) is often hindered by the black-box nature of deep neural networks. This lack of transparency poses significant challenges for regulatory compliance and human-agent trust. This paper presents an experimental study aimed at making high-performance continuous control DRL systems interpretable. A policy distillation framework is implemented using the classic Inverted Pendulum benchmark. A high-performance Twin Delayed DDPG (TD3) agent serves as an opaque, continuous teacher model, whose policy is distilled into an interpretable student surrogate based on a shallow Decision Tree. By leveraging a custom physics-aware feature and "Noisy Oracle Rollouts" for dataset generation, the distillation process achieves performance equivalent to the expert teacher. Furthermore, comparative control theory analysis reveals a fundamental trade-off: transitioning from continuous to discrete rule-based control induces high-frequency Bang-Bang actuation and a stable bimodal limit cycle. Simulation results indicate that Bounded-Input Bounded-Output (BIBO) stability is maintained while providing both global and local interpretability for safe autonomous systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。