用轻量级元控制器提升大规模网络攻防博弈的决策效率
A Scalable Approach to Solving Simulation-Based Network Security Games
- 通过结构嵌入投影和筛选层，快速定位关键设备进行集中策略搜索
- 在大型网络上实现更高收益，且内存与训练时间增长可控
- 适合需要高效处理大规模分布式系统安全决策的研究者
我们提出 MetaDOAR，一种轻量级元控制器，增强双盲对弈/基于策略的自洽优化（PSRO）范式，引入学习型分区感知过滤层与Q值缓存，以实现超大规模网络环境下的多智能体强化学习。MetaDOAR从每个节点的结构嵌入中学习紧凑状态投影，快速评分并选择一小部分设备（即 top-k 分区），由低层级智能体在此子集上执行聚焦的束搜索，借助评判器代理完成评估。候选动作通过批量评判前向传播计算，并存储于以量化状态投影和本地动作标识为键的LRU缓存中，通过保守的k跳缓存失效策略显著减少重复评判计算，同时保持决策质量。实验证明，MetaDOAR在大型网络拓扑上优于当前最优基线，且在内存占用与训练时间方面无明显扩展瓶颈。该工作为大规模网络化决策问题提供了可实践、有理论依据的分层策略学习路径。
原文摘要 · Abstract (English)
We introduce MetaDOAR, a lightweight meta-controller that augments the Double Oracle / PSRO paradigm with a learned, partition-aware filtering layer and Q-value caching to enable scalable multi-agent reinforcement learning on very large cyber-network environments. MetaDOAR learns a compact state projection from per node structural embeddings to rapidly score and select a small subset of devices (a top-k partition) on which a conventional low-level actor performs focused beam search utilizing a critic agent. Selected candidate actions are evaluated with batched critic forwards and stored in an LRU cache keyed by a quantized state projection and local action identifiers, dramatically reducing redundant critic computation while preserving decision quality via conservative k-hop cache invalidation. Empirically, MetaDOAR attains higher player payoffs than SOTA baselines on large network topologies, without significant scaling issues in terms of memory usage or training time. This contribution provide a practical, theoretically motivated path to efficient hierarchical policy learning for large-scale networked decision problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。