用强化学习在线优化调度,让系统实时应对突发情况
Adaptive Approach to Enhance Machine Learning Scheduling Algorithms During Runtime Using Reinforcement Learning in Metascheduling Applications
- 在元调度器中嵌入强化学习模块,实现在线动态优化
- 通过持续探索发现新调度方案,逐步扩展调度图覆盖范围
- 适合需要高可靠性的实时系统,如工业控制与安全关键场景
在时间触发架构中,元调度对适应动态不可预测环境至关重要,确保任务执行的可靠性和效率。然而,传统离线训练人工智能调度推理方法面临挑战,尤其在于构建涵盖所有可能场景的完整多调度图(MSG)时的复杂性。生成能覆盖广泛概率空间的MSG,特别是在考虑硬件故障、空闲时间变化或模式切换等上下文事件时,资源消耗巨大且常不可行。为此,我们提出在元调度器中集成自适应在线学习单元,以提升实时性能。其核心动机源于离线训练的局限性——所建MSG本质上仅为完整空间的子集,仅聚焦最可能和最关键的上下文事件。在线模式下,强化学习(RL)通过持续探索与发现新调度方案,不断扩展MSG并提升系统性能。该动态适应机制使系统能更有效地应对意外事件与复杂调度场景。在在线学习单元中实现了多个RL模型,各自针对特定调度挑战设计,不仅促进新解的发现,也优化现有调度器,尤其在引入更严格截止时间或新性能标准时。通过实时训练持续优化AI推理,系统保持灵活性,可满足不断演进的需求,从而保障大规模、安全关键环境中的鲁棒性与高效性。
原文摘要 · Abstract (English)
Metascheduling in time-triggered architectures has been crucial in adapting to dynamic and unpredictable environments, ensuring the reliability and efficiency of task execution. However, traditional approaches face significant challenges when training Artificial Intelligence (AI) scheduling inferences offline, particularly due to the complexities involved in constructing a comprehensive Multi-Schedule Graph (MSG) that accounts for all possible scenarios. The process of generating an MSG that captures the vast probability space, especially when considering context events like hardware failures, slack variations, or mode changes, is resource-intensive and often infeasible. To address these challenges, we propose an adaptive online learning unit integrated within the metascheduler to enhance performance in real-time. The primary motivation for developing this unit stems from the limitations of offline training, where the MSG created is inherently a subset of the complete space, focusing only on the most probable and critical context events. In the online mode, Reinforcement Learning (RL) plays a pivotal role by continuously exploring and discovering new scheduling solutions, thus expanding the MSG and enhancing system performance over time. This dynamic adaptation allows the system to handle unexpected events and complex scheduling scenarios more effectively. Several RL models were implemented within the online learning unit, each designed to address specific challenges in scheduling. These models not only facilitate the discovery of new solutions but also optimize existing schedulers, particularly when stricter deadlines or new performance criteria are introduced. By continuously refining the AI inferences through real-time training, the system remains flexible and capable of meeting evolving demands, thus ensuring robustness and efficiency in large-scale, safety-critical environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。