用强化学习让探针机器人识别集群领袖,还能跨场景通用。
On Swarm Leader Identification using Probing Policies
- 设计交互式探针策略,通过物理接触探测集群成员来定位领袖。
- 在不同规模和速度的集群中识别准确率超基准模型,零样本泛化能力强。
- 适合需要隐蔽领袖的对抗性任务,如军事或安全巡检场景。
在对抗性环境中,识别并隐藏集群领袖对任务成功至关重要。本文提出交互式集群领袖识别(iSLI)问题,即由一个对抗性探针代理通过与集群成员的物理互动来识别其领袖。将iSLI建模为部分可观测马尔可夫决策过程(POMDP),采用近端策略优化(PPO)训练探针策略。提出一种新型神经网络架构,结合时序图关系变换器(TGR)层与简化结构状态空间序列(S5)模型。TGR层有效处理基于图的集群观测,利用学习到的门控机制捕捉时间依赖性和融合关系信息,生成利于策略学习的表示。大量仿真表明,所提TGR模型优于基线图神经网络架构,并在训练外的集群规模与速度条件下表现出显著零样本泛化能力。训练后的探针在多种分布外场景下仍保持高识别准确率,并具备合理的置信度。真实机器人实验进一步验证了该方法从仿真到现实的迁移效果,以及对突发断连等动态变化的鲁棒性。
原文摘要 · Abstract (English)
Identifying the leader within a robotic swarm is crucial, especially in adversarial contexts where leader concealment is necessary for mission success. This work introduces the interactive Swarm Leader Identification (iSLI) problem, a novel approach where an adversarial probing agent identifies a swarm's leader by physically interacting with its members. We formulate the iSLI problem as a Partially Observable Markov Decision Process (POMDP) and employ Deep Reinforcement Learning, specifically Proximal Policy Optimization (PPO), to train the prober's policy. The proposed approach utilizes a novel neural network architecture featuring a Timed Graph Relationformer (TGR) layer combined with a Simplified Structured State Space Sequence (S5) model. The TGR layer effectively processes graph-based observations of the swarm, capturing temporal dependencies and fusing relational information using a learned gating mechanism to generate informative representations for policy learning. Extensive simulations demonstrate that our TGR-based model outperforms baseline graph neural network architectures and exhibits significant zero-shot generalization capabilities across varying swarm sizes and speeds different from those used during training. The trained prober achieves high accuracy in identifying the leader, maintaining performance even in out-of-training distribution scenarios, and showing appropriate confidence levels in its predictions. Real-world experiments with physical robots further validate the approach, confirming successful sim-to-real transfer and robustness to dynamic changes, such as unexpected agent disconnections.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。