数字孪生加速多智能体在线学习，减少系统变化后的试错成本。
TwinLoop: Simulation-in-the-Loop Digital Twins for Online Multi-Agent Reinforcement Learning
- 用数字孪生模拟环境，提前优化策略再同步到真实系统
- 在负载和基础设施变化后，适应速度提升显著，试错次数减少
- 适合需要快速响应的车联网、工业控制等实时系统
去中心化在线学习可实现网络物理多智能体系统的运行时自适应，但当运行条件变化时，学习到的策略往往需大量试错才能恢复性能。为此，我们提出TwinLoop——一种面向在线多智能体强化学习的仿真-闭环数字孪生框架。当发生上下文切换时，数字孪生被触发以重建当前系统状态，从最新智能体策略出发，通过仿真‘如果’分析加速策略优化，再将更新参数同步回物理系统中的智能体。我们在动态负载与基础设施变化的车载边缘计算任务卸载场景中评估了TwinLoop。结果表明，数字孪生能有效提升变化后的适应效率，降低对昂贵在线试错的依赖。
原文摘要 · Abstract (English)
Decentralised online learning enables runtime adaptation in cyber-physical multi-agent systems, but when operating conditions change, learned policies often require substantial trial-and-error interaction before recovering performance. To address this, we propose TwinLoop, a simulation-in-the-loop digital twin framework for online multi-agent reinforcement learning. When a context shift occurs, the digital twin is triggered to reconstruct the current system state, initialise from the latest agent policies, and perform accelerated policy improvement with simulation what-if analysis before synchronising updated parameters back to the agents in the physical system. We evaluate TwinLoop in a vehicular edge computing task-offloading scenario with changing workload and infrastructure conditions. The results suggest that digital twins can improve post-shift adaptation efficiency and reduce reliance on costly online trial-and-error.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。