用概率神经网络检测强化学习智能体的异常状态，提升网络安全自治系统的可信度。
Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agents
- 基于概率神经网络构建异常检测机制，识别强化学习代理在离域状态下的异常行为。
- 在模拟环境中测试多种攻击策略，验证了算法在大量实验中的高效性。
- 适用于需要可靠风险识别的网络安全自治系统，尤其适合融合符号与学习组件的智能体。
面向网络安全应用的自主智能体通过结合传统与学习型组件，利用强化学习（RL）算法实现自适应防御、推理和安全规则部署，同时维持关键运行流程。然而，训练阶段对网络状态及环境的知识可能有限，导致智能体难以应对未知情况。为确保其可信性，需能可靠检测无法处理的异常情境并移交至网络安全专家。本文提出一种基于概率神经网络（PNN）的分布外（OOD）监测算法，用于检测具有离散状态与动作的基于强化学习的智能体所面临的异常或离域情况。为验证有效性，将该算法集成至使用学习型组件的行为树框架的神经符号自治网络代理中，并在模拟网络环境中针对不同对抗策略进行评估。大量实验结果表明，所提方法整体上具有高效率。
原文摘要 · Abstract (English)
Autonomous agents for cyber applications take advantage of modern defense techniques by adopting intelligent agents with conventional and learning-enabled components. These intelligent agents are trained via reinforcement learning (RL) algorithms, and can learn, adapt to, reason about and deploy security rules to defend networked computer systems while maintaining critical operational workflows. However, the knowledge available during training about the state of the operational network and its environment may be limited. The agents should be trustworthy so that they can reliably detect situations they cannot handle, and hand them over to cyber experts. In this work, we develop an out-of-distribution (OOD) Monitoring algorithm that uses a Probabilistic Neural Network (PNN) to detect anomalous or OOD situations of RL-based agents with discrete states and discrete actions. To demonstrate the effectiveness of the proposed approach, we integrate the OOD monitoring algorithm with a neurosymbolic autonomous cyber agent that uses behavior trees with learning-enabled components. We evaluate the proposed approach in a simulated cyber environment under different adversarial strategies. Experimental results over a large number of episodes illustrate the overall efficiency of our proposed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。