为多智能体强化学习系统提供可解释的故障溯源方法。
Interpretable Failure Analysis in Multi-Agent Reinforcement Learning Systems
- 基于梯度的两阶段框架,通过泰勒余项分析定位初始故障源。
- 在500次实验中检测准确率达88.2%-99.4%,并揭示故障传播路径。
- 适合安全关键领域中需要故障可解释性的研究人员与工程师。
多智能体强化学习(MARL)正日益应用于安全关键领域,但可解释的故障检测与归因方法仍不完善。本文提出一种两阶段梯度驱动框架,用于解决三大关键故障分析任务:(1) 检测真正的初始故障源(Patient-0);(2) 验证非攻击智能体为何因连锁效应被率先标记;(3) 追踪故障如何通过学习到的协作路径传播。第一阶段通过策略梯度成本的泰勒余项分析实现可解释的单智能体故障检测,在首次阈值交叉时确定候选Patient-0。第二阶段通过价值函数导数的一阶敏感性与因果窗口内方向二阶曲率的几何分析,构建可解释的传染图谱。该方法通过揭示放大上游偏差的路径,解释了“下游优先”检测异常现象。在Simple Spread(3和5智能体)的500个回合及StarCraft II中100个回合的MADDPG和HATRPO实验中,该方法实现了88.2%-99.4%的Patient-0检测准确率,并提供可解释的几何证据支持决策。本框架从黑箱检测转向梯度级可解释取证,为安全关键MARL系统的级联故障诊断提供了实用工具。
原文摘要 · Abstract (English)
Multi-Agent Reinforcement Learning (MARL) is increasingly deployed in safety-critical domains, yet methods for interpretable failure detection and attribution remain underdeveloped. We introduce a two-stage gradient-based framework that provides interpretable diagnostics for three critical failure analysis tasks: (1) detecting the true initial failure source (Patient-0); (2) validating why non-attacked agents may be flagged first due to domino effects; and (3) tracing how failures propagate through learned coordination pathways. Stage 1 performs interpretable per-agent failure detection via Taylor-remainder analysis of policy-gradient costs, declaring an initial Patient-0 candidate at the first threshold crossing. Stage 2 provides validation through geometric analysis of critic derivatives-first-order sensitivity and directional second-order curvature aggregated over causal windows to construct interpretable contagion graphs. This approach explains "downstream-first" detection anomalies by revealing pathways that amplify upstream deviations. Evaluated across 500 episodes in Simple Spread (3 and 5 agents) and 100 episodes in StarCraft II using MADDPG and HATRPO, our method achieves 88.2-99.4% Patient-0 detection accuracy while providing interpretable geometric evidence for detection decisions. By moving beyond black-box detection to interpretable gradient-level forensics, this framework offers practical tools for diagnosing cascading failures in safety-critical MARL systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。