arXiv:2412.19311cs.AIcs.HC2024-12中稿 · 24th International…被引 4

让强化学习更安全可信,通过解释能力发现并修复决策漏洞。

xSRL: Safety-Aware Explainable Reinforcement Learning -- Safety as a Product of Explainability

  • 融合局部与全局解释,动态分析强化学习决策过程
  • 可识别策略漏洞并支持无重训练修复,提升系统可靠性
  • 适合自动驾驶、机器人等高风险场景的开发与验证

强化学习在模拟环境(如游戏)中表现优异,但在自动驾驶、机器人、无人机和医疗设备等真实系统中部署时,必须兼顾安全性与透明性,尤其面对对抗威胁。现有安全强化学习方法虽能优化任务性能与安全约束,但错误仍不可避免,此时解释能力对建立人类信任至关重要。当前可解释性方法难以应对强化学习的动态序列特性及性能与安全的长期权衡。传统机器学习解释工具(如显著图)在安全关键场景中效果不足。为此,我们提出xSRL框架,集成局部与全局解释,全面理解强化学习代理行为,并支持通过对抗攻击识别策略漏洞,提供无需重训练的调试与修补工具。实验与用户研究证明,xSRL显著提升强化学习系统的安全性,使其更可靠、可信,适用于真实世界部署。代码已开源:https://github.com/risal-shefin/xSRL。

原文摘要 · Abstract (English)

Reinforcement learning (RL) has shown great promise in simulated environments, such as games, where failures have minimal consequences. However, the deployment of RL agents in real-world systems such as autonomous vehicles, robotics, UAVs, and medical devices demands a higher level of safety and transparency, particularly when facing adversarial threats. Safe RL algorithms have been developed to address these concerns by optimizing both task performance and safety constraints. However, errors are inevitable, and when they occur, it is essential that the RL agents can also explain their actions to human operators. This makes trust in the safety mechanisms of RL systems crucial for effective deployment. Explainability plays a key role in building this trust by providing clear, actionable insights into the agent's decision-making process, ensuring that safety-critical decisions are well understood. While machine learning (ML) has seen significant advances in interpretability and visualization, explainability methods for RL remain limited. Current tools fail to address the dynamic, sequential nature of RL and its needs to balance task performance with safety constraints over time. The re-purposing of traditional ML methods, such as saliency maps, is inadequate for safety-critical RL applications where mistakes can result in severe consequences. To bridge this gap, we propose xSRL, a framework that integrates both local and global explanations to provide a comprehensive understanding of RL agents' behavior. xSRL also enables developers to identify policy vulnerabilities through adversarial attacks, offering tools to debug and patch agents without retraining. Our experiments and user studies demonstrate xSRL's effectiveness in increasing safety in RL systems, making them more reliable and trustworthy for real-world deployment. Code is available at https://github.com/risal-shefin/xSRL.

强化学习可解释性安全机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。