用强化学习优化工厂装配线调度,提速且更高效。
Novel Multi-Agent Action Masked Deep Reinforcement Learning for General Industrial Assembly Lines Balancing Problems
- 将装配线建模为马尔可夫决策过程,不依赖特定产线类型。
- 引入动作掩码和多智能体架构,训练速度更快,状态空间更小。
- 适合大规模工业调度场景,支持实时决策部署。
高效的任务规划对现代工业装配线维持制造标准、避免项目约束违规及实现成本效益至关重要。尽管整数规划(IP)可获得精确解,但其搜索空间随输入参数增长常导致大规模场景下计算不可行;遗传算法等启发式方法虽可用,但在复杂情况下常得次优解。本文提出一种通用装配线的新型数学模型,将其表述为马尔可夫决策过程(MDP),无需对产线类型做假设,显著区别于现有多数模型。该模型用于构建虚拟环境,训练深度强化学习(DRL)智能体以优化任务与资源调度。为提升训练效率,提出两项创新:一是动作掩码技术,确保智能体仅选择可行动作,减少无效探索;二是多智能体结构,每个工位由独立智能体管理,从而缩小状态与动作空间。采用集中训练、分散执行框架,构建可扩展的学习架构,使智能体离线学习后,可通过神经网络将当前工厂状态映射至最优动作,实现实时决策。数值仿真验证了该方案的有效性,相比传统模型方法收敛速度显著提升。
原文摘要 · Abstract (English)
Efficient planning of activities is essential for modern industrial assembly lines to uphold manufacturing standards, prevent project constraint violations, and achieve cost-effective operations. While exact solutions to such challenges can be obtained through Integer Programming (IP), the dependence of the search space on input parameters often makes IP computationally infeasible for large-scale scenarios. Heuristic methods, such as Genetic Algorithms, can also be applied, but they frequently produce suboptimal solutions in extensive cases. This paper introduces a novel mathematical model of a generic industrial assembly line formulated as a Markov Decision Process (MDP), without imposing assumptions on the type of assembly line a notable distinction from most existing models. The proposed model is employed to create a virtual environment for training Deep Reinforcement Learning (DRL) agents to optimize task and resource scheduling. To enhance the efficiency of agent training, the paper proposes two innovative tools. The first is an action-masking technique, which ensures the agent selects only feasible actions, thereby reducing training time. The second is a multi-agent approach, where each workstation is managed by an individual agent, as a result, the state and action spaces were reduced. A centralized training framework with decentralized execution is adopted, offering a scalable learning architecture for optimizing industrial assembly lines. This framework allows the agents to learn offline and subsequently provide real-time solutions during operations by leveraging a neural network that maps the current factory state to the optimal action. The effectiveness of the proposed scheme is validated through numerical simulations, demonstrating significantly faster convergence to the optimal solution compared to a comparable model-based approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。