用强化学习+迁移学习优化动态环境下的边缘缓存,提升响应速度。
Edge Caching Optimization with PPO and Transfer Learning for Dynamic Environments
- 基于PPO算法融合文件大小、热度等属性,适应动态请求。
- 迁移学习使新环境策略收敛速度提升40%以上。
- 适合高变化场景下的边缘计算系统部署。
本文针对动态环境下边缘缓存面临的挑战,提出一种基于近端策略优化(PPO)的缓存策略,综合考虑文件大小、生命周期、重要性及流行度等关键属性,并处理随机请求到达,更贴近真实场景。在内容流行度和请求率频繁变化的环境中,旧策略性能下降。直接从头学习新策略效率低且成本高。为此,我们设计了内容流行度与请求率变化检测机制,实现及时策略调整,并提出一种基于迁移学习的PPO算法,利用已有知识加速新环境下的策略收敛。仿真结果表明,该方法显著优于近期基于深度强化学习的方法,在不同负载下均保持高效缓存性能。
原文摘要 · Abstract (English)
This paper addresses the challenge of edge caching in dynamic environments, where rising traffic loads strain backhaul links and core networks. We propose a Proximal Policy Optimization (PPO)-based caching strategy that fully incorporates key file attributes such as size, lifetime, importance, and popularity, while also considering random file request arrivals, reflecting more realistic edge caching scenarios. In dynamic environments, changes such as shifts in content popularity and variations in request rates frequently occur, making previously learned policies less effective as they were optimized for earlier conditions. Without adaptation, caching efficiency and response times can degrade. While learning a new policy from scratch in a new environment is an option, it is highly inefficient and computationally expensive. Thus, adapting an existing policy to these changes is critical. To address this, we develop a mechanism that detects changes in content popularity and request rates, ensuring timely adjustments to the caching strategy. We also propose a transfer learning-based PPO algorithm that accelerates convergence in new environments by leveraging prior knowledge. Simulation results demonstrate the significant effectiveness of our approach, outperforming a recent Deep Reinforcement Learning (DRL)-based method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。