针对边缘计算场景优化强化学习，提升效率并降低资源消耗。
Edge Delayed Deep Deterministic Policy Gradient: efficient continuous control for edge scenarios
- 基于DDPG改进，引入延迟更新机制减少过估计偏差。
- 比现有方法节省30%计算资源，GPU耗时减少25%。
- 适合隐私敏感、算力受限的边缘智能应用。
深度强化学习因其在高维环境中的复杂策略学习能力而受到关注。近期进展采用双网络架构通过Q-learning算法学习最优策略，但存在过高估计偏差问题，可能干扰学习过程并降低策略性能。为此，新算法通过多Q函数设计缓解过估计偏差。边缘场景因注重隐私而日益重要，但有限的计算资源对复杂机器学习方法构成挑战，算法效率至关重要。本文提出专为边缘场景设计的新强化学习算法——边缘延迟深度确定性策略梯度（EdgeD3），该算法在不增加内存占用的前提下，使深度确定性策略梯度（DDPG）的性能显著提升，实现25%的GPU时间节省。同时，EdgeD3在多个基准测试中始终达到或超越当前最优方法表现，且计算资源使用减少30%，内存需求降低30%。
原文摘要 · Abstract (English)
Deep Reinforcement Learning is gaining increasing attention thanks to its capability to learn complex policies in high-dimensional settings. Recent advancements utilize a dual-network architecture to learn optimal policies through the Q-learning algorithm. However, this approach has notable drawbacks, such as an overestimation bias that can disrupt the learning process and degrade the performance of the resulting policy. To address this, novel algorithms have been developed that mitigate overestimation bias by employing multiple Q-functions. Edge scenarios, which prioritize privacy, have recently gained prominence. In these settings, limited computational resources pose a significant challenge for complex Machine Learning approaches, making the efficiency of algorithms crucial for their performance. In this work, we introduce a novel Reinforcement Learning algorithm tailored for edge scenarios, called Edge Delayed Deep Deterministic Policy Gradient (EdgeD3). EdgeD3 enhances the Deep Deterministic Policy Gradient (DDPG) algorithm, achieving significantly improved performance with $25\%$ less Graphics Process Unit (GPU) time while maintaining the same memory usage. Additionally, EdgeD3 consistently matches or surpasses the performance of state-of-the-art methods across various benchmarks, all while using $30\%$ fewer computational resources and requiring $30\%$ less memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。