用强化学习+区块链实现无人机网络的安全高效路由
Trusted Routing for Blockchain-Empowered UAV Networks via Multi-Agent Deep Reinforcement Learning
- 多智能体深度强化学习处理动态网络下的分布式路由决策
- 引入区块链信任机制，识别低可信无人机并降低延迟13%以上
- 适合研究无人机安全通信与分布式系统可信控制的读者
由于高灵活性和多功能性，无人机（UAV）被广泛应用于监控和灾后救援等领域。然而，在分布式拓扑和高度动态的无人机网络中，路由易受恶意节点攻击，保障路由安全极具挑战。本文针对存在恶意节点的时间变动态无人机网络，建模了路由问题以最小化总延迟，该问题是整数线性规划且难以求解。为应对安全问题，设计了一种基于区块链的信任管理机制（BTMM），可动态评估信任值并识别低信任度无人机。针对传统实用拜占庭容错算法在区块链中的不足，提出共识无人机更新机制。考虑到局部可观测性，将路由问题重构为去中心化的部分可观测马尔可夫决策过程。进一步设计基于多智能体双深度Q网络的路由算法以最小化总延迟。仿真结果表明，在存在攻击无人机的情况下，所提方法相比多智能体近端策略优化算法、多智能体深度Q网络算法以及无BTMM的方法，延迟分别降低了13.39%、12.74%和16.6%。
原文摘要 · Abstract (English)
Due to the high flexibility and versatility, unmanned aerial vehicles (UAVs) are leveraged in various fields including surveillance and disaster rescue.However, in UAV networks, routing is vulnerable to malicious damage due to distributed topologies and high dynamics. Hence, ensuring the routing security of UAV networks is challenging. In this paper, we characterize the routing process in a time-varying UAV network with malicious nodes. Specifically, we formulate the routing problem to minimize the total delay, which is an integer linear programming and intractable to solve. Then, to tackle the network security issue, a blockchain-based trust management mechanism (BTMM) is designed to dynamically evaluate trust values and identify low-trust UAVs. To improve traditional practical Byzantine fault tolerance algorithms in the blockchain, we propose a consensus UAV update mechanism. Besides, considering the local observability, the routing problem is reformulated into a decentralized partially observable Markov decision process. Further, a multi-agent double deep Q-network based routing algorithm is designed to minimize the total delay. Finally, simulations are conducted with attacked UAVs and numerical results show that the delay of the proposed mechanism decreases by 13.39$\%$, 12.74$\%$, and 16.6$\%$ than multi-agent proximal policy optimal algorithms, multi-agent deep Q-network algorithms, and methods without BTMM, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。