对比强化学习方法在真实晶圆厂调度中的可扩展性,发现进化策略更优。
Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets
- 用进化策略替代策略梯度法优化晶圆厂调度
- 真实数据下延误降低4%、吞吐率提升1%,开源数据提升超10%
- 多场景训练数据提升泛化能力,适合工业级调度研究者
基准数据集对半导体前端晶圆厂调度方法的开发与部署至关重要。然而,常用基准如Minifab或SMT2020缺乏真实场景的复杂细节与约束。为弥补这一不足,我们对比开源仿真模型与真实工业数据集,评估优化方法在不同复杂度下的可扩展性。重点研究基于策略梯度和进化策略的强化学习方法。结果显示,所提出的进化策略方法在复杂度增加时表现出更强的可扩展性;关键在于让智能体控制相关瓶颈设备。利用多样化训练数据,在不同负载与随机设备故障场景下均实现良好泛化。尽管整体计算成本高,但训练可随CPU核心数良好扩展。在真实工业数据集上,延误最多降低4%,吞吐率提升1%;在较简单的Minifab与SMT2020数据集上,延误改善达双位数百分比,吞吐率提升个位数百分比。
原文摘要 · Abstract (English)
Benchmark datasets are crucial for evaluating approaches to scheduling or dispatching in the semiconductor industry during the development and deployment phases. However, commonly used benchmark datasets like the Minifab or SMT2020 lack the complex details and constraints found in real-world scenarios. To mitigate this shortcoming, we compare open-source simulation models with a real industry dataset to evaluate how optimization methods scale with different levels of complexity. Specifically, we focus on Reinforcement Learning methods, performing optimization based on policy-gradient and Evolution Strategies. Our research provides insights into the effectiveness of these optimization methods and their applicability to realistic semiconductor frontend fab simulations. We show that our proposed Evolution Strategies-based method scales much better than a comparable policy-gradient-based approach. Moreover, we identify the selection and combination of relevant bottleneck tools to control by the agent as crucial for an efficient optimization. For the generalization across different loading scenarios and stochastic tool failure patterns, we achieve advantages when utilizing a diverse training dataset. While the overall approach is computationally expensive, it manages to scale well with the number of CPU cores used for training. For the real industry dataset, we achieve an improvement of up to 4% regarding tardiness and up to 1% regarding throughput. For the less complex open-source models Minifab and SMT2020, we observe double-digit percentage improvement in tardiness and single digit percentage improvement in throughput by use of Evolution Strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。