提出快速自适应的强化学习调度方法,解决边缘计算中实时任务的动态调度难题。
Agile Reinforcement Learning for Real-Time Task Scheduling in Edge Computing
- 采用智能探索与动作掩码机制,只选择相关动作,减少无效搜索。
- 在多个场景下达到更高任务命中率,收敛速度比基线快3倍以上。
- 适合需要快速响应的软实时应用,如自动驾驶、远程医疗等边缘场景。
软实时应用日益复杂,在边缘计算环境中调度迁移任务面临严峻挑战,需满足任务时序约束。同时,搜索空间呈指数增长、多目标与参数并存,以及边缘环境的高度动态性进一步加剧了调度复杂性。传统启发式与元启发式算法因难以适应动态环境和复杂特征,常无法生成最优或近似最优调度方案。为此,研究引入强化学习以应对边缘计算任务调度中的复杂性与动态性。然而,强化学习存在学习时间长的问题,主要源于庞大的全局动作空间及频繁的无关动作随机探索。本文提出敏捷强化学习(aRL),通过引导探索与动作掩码机制,使智能体仅执行相关动作,提升预测性,实现快速适应与收敛。实验表明,结合智能探索与动作掩码的方法使aRL在多个测试场景中显著提升任务命中率,并在较短时间内完成收敛,优于现有基线方法。
原文摘要 · Abstract (English)
Soft real-time applications are becoming increasingly complex, posing significant challenges for scheduling offloaded tasks in edge computing environments while meeting task timing constraints. Moreover, the exponential growth of the search space, presence of multiple objectives and parameters, and highly dynamic nature of edge computing environments further exacerbate the complexity of task scheduling. As a result, schedulers based on heuristic and metaheuristic algorithms frequently encounter difficulties in generating optimal or near-optimal task schedules due to their constrained ability to adapt to the dynamic conditions and complex environmental characteristics of edge computing. Accordingly, reinforcement learning algorithms have been incorporated into schedulers to address the complexity and dynamic conditions inherent in task scheduling in edge computing. However, a significant limitation of reinforcement learning algorithms is the prolonged learning time required to adapt to new environments and to address medium- and large-scale problems. This challenge arises from the extensive global action space and frequent random exploration of irrelevant actions. Therefore, this study proposes Agile Reinforcement learning (aRL), in which the RL-agent performs informed exploration and executes only relevant actions. Consequently, the predictability of the RL-agent is enhanced, leading to rapid adaptation and convergence, which positions aRL as a suitable candidate for scheduling the tasks of soft real-time applications in edge computing. The experiments demonstrate that the combination of informed exploration and action-masking methods enables aRL to achieve a higher hit-ratio and converge faster than the baseline approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。