通过分析攻防行为事件,揭示强化学习防御代理的成败机制。
Interpreting Agent Behaviors in Reinforcement-Learning-Based Cyber-Battle Simulation Platforms
- 简化状态与动作空间,追踪关键事件以解析攻防行为
- 发现防御方在1-2个时间步内可清除多数入侵,部分操作无效率达40%-99%
- 揭示诱饵服务可阻断高达94%的特权访问攻击,适合安全策略设计者参考
我们分析了提交至CAGE挑战赛的两个开源深度强化学习防御代理,该挑战赛要求代理在模拟网络中抵御多种规则基础的攻击代理。通过简化复杂的状态与动作空间,并追踪关键事件,我们揭示了攻防双方在不同实验场景中的细粒度行为模式。分析表明,防御方通常能在主机被利用后1-2个时间步内完成清理;通过考察环境状态变化,发现部分关键操作有效性仅为40%至99%;此外,诱饵服务可阻断高达94%的直接获取主机特权的攻击。最后,我们讨论了挑战的现实性,并指出CAGE Challenge 4已针对部分问题进行了改进。
原文摘要 · Abstract (English)
We analyze two open source deep reinforcement learning agents submitted to the CAGE Challenge 2 cyber defense challenge, where each competitor submitted an agent to defend a simulated network against each of several provided rules-based attack agents. We demonstrate that one can gain interpretability of agent successes and failures by simplifying the complex state and action spaces and by tracking important events, shedding light on the fine-grained behavior of both the defense and attack agents in each experimental scenario. By analyzing important events within an evaluation episode, we identify patterns in infiltration and clearing events that tell us how well the attacker and defender played their respective roles; for example, defenders were generally able to clear infiltrations within one or two timesteps of a host being exploited. By examining transitions in the environment's state caused by the various possible actions, we determine which actions tended to be effective and which did not, showing that certain important actions are between 40% and 99% ineffective. We examine how decoy services affect exploit success, concluding for instance that decoys block up to 94% of exploits that would directly grant privileged access to a host. Finally, we discuss the realism of the challenge and ways that the CAGE Challenge 4 has addressed some of our concerns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。