解决移动用户在边缘网络中数字孪生的同步与迁移难题，降低能耗。
Two-Timescale Synchronization and Migration for Digital Twin Networks: A Multi-Agent Deep Reinforcement Learning Approach
- 分时尺度设计同步与迁移机制，兼顾实时性与可靠性。
- 提出Beta-HAPPO算法，使用户能耗降低显著优于基准方法。
- 适合研究边缘计算与数字孪生系统融合的工程师和学者。
数字孪生（DT）作为实时呈现物理世界状态和实现自维持系统的关键技术，常部署于多接入边缘计算（MEC）网络中以降低延迟。为保证数字孪生的准确性，移动用户（MU）需定期与自身孪生体同步状态。然而，用户移动带来两大挑战：一是移动引发的数字孪生迁移可能导致同步失败；二是频繁同步虽能保障孪生体保真度，但迁移却可能不频繁。为此，本文建立非凸随机优化问题，旨在最小化用户的长期平均能耗，并通过李雅普诺夫理论处理可靠性约束，将问题转化为部分可观测马尔可夫决策过程（POMDP）。进一步提出异构代理近端策略优化结合贝塔分布（Beta-HAPPO）方法求解。数值结果表明，所提方法在能耗节约方面显著优于其他基准方案。
原文摘要 · Abstract (English)
Digital twins (DTs) have emerged as a promising enabler for representing the real-time states of physical worlds and realizing self-sustaining systems. In practice, DTs of physical devices, such as mobile users (MUs), are commonly deployed in multi-access edge computing (MEC) networks for the sake of reducing latency. To ensure the accuracy and fidelity of DTs, it is essential for MUs to regularly synchronize their status with their DTs. However, MU mobility introduces significant challenges to DT synchronization. Firstly, MU mobility triggers DT migration which could cause synchronization failures. Secondly, MUs require frequent synchronization with their DTs to ensure DT fidelity. Nonetheless, DT migration among MEC servers, caused by MU mobility, may occur infrequently. Accordingly, we propose a two-timescale DT synchronization and migration framework with reliability consideration by establishing a non-convex stochastic problem to minimize the long-term average energy consumption of MUs. We use Lyapunov theory to convert the reliability constraints and reformulate the new problem as a partially observable Markov decision-making process (POMDP). Furthermore, we develop a heterogeneous agent proximal policy optimization with Beta distribution (Beta-HAPPO) method to solve it. Numerical results show that our proposed Beta-HAPPO method achieves significant improvements in energy savings when compared with other benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。