让强化学习在模拟中学会应对网络延迟，提升真实部署表现。
CALF: Communication-Aware Learning Framework for Distributed Reinforcement Learning
- 在训练时模拟真实网络条件，显式建模延迟和丢包。
- 相比传统方法，部署性能差距显著缩小，实测效果更稳定。
- 适合边缘计算与云协同的分布式强化学习应用者。
分布式强化学习在边缘设备与云服务器间部署时，会遭遇网络延迟、抖动和丢包问题。标准RL训练假设零延迟交互，导致在真实网络环境下性能严重下降。我们提出通信感知学习框架CALF（Communication-Aware Learning Framework），在仿真中引入真实网络模型进行策略训练。系统性实验表明，采用网络感知训练能显著缩小部署性能差距，优于无网络感知基线。在异构硬件上的分布式策略部署验证了：训练阶段显式建模通信约束，可实现鲁棒的真实世界执行。研究确立了网络条件作为类Wi-Fi分布式部署中模拟到现实迁移的关键维度，补充了物理与视觉域随机化的作用。
原文摘要 · Abstract (English)
Distributed reinforcement learning policies face network delays, jitter, and packet loss when deployed across edge devices and cloud servers. Standard RL training assumes zero-latency interaction, causing severe performance degradation under realistic network conditions. We introduce CALF (Communication-Aware Learning Framework), which trains policies under realistic network models during simulation. Systematic experiments demonstrate that network-aware training substantially reduces deployment performance gaps compared to network-agnostic baselines. Distributed policy deployments across heterogeneous hardware validate that explicitly modelling communication constraints during training enables robust real-world execution. These findings establish network conditions as a major axis of sim-to-real transfer for Wi-Fi-like distributed deployments, complementing physics and visual domain randomisation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。