Abstract: To solve a collaborative manufacturing scheduling problem where the Dominant Manufacturer (DM) outsources several processes to Collaborative Manufacturers (CMs) with the objective of minimizing overall costs for different orders,a Deep Reinforcement Learning (DRL) framework was proposed,which integrated disjunctive graph analysis to tackle the intricate scheduling dynamics inherent in collaborative manufacturing networks.In this way,the Agent learns the action strategy based on the input order status.The scheduling problem was transformed into a sequential decision-making task by employing a two-dimensional action space derived from the disjunctive graph structure.Through setting the manufacturing state as the input of a deep neural network model,the collaborative scheduling problem was transformed into a Markov Decision Process (MDP) problem.Additionally,a cost-oriented reward function was formulated to guide the exploration process of the agent aiming to identify the optimal action.The experimental results showed that the deep reinforcement learning algorithm outperformed any single scheduling rule,and also performed more superiorly on average in terms of solution effectiveness when compared to genetic algorithms.

Key words: collaborative manufacturing, disjunctive graph, deep reinforcement learning, scheduling, Markov decision process

摘要： 针对主导制造商将多个流程外包给协同制造商的协同调度问题,考虑同类产品不同订单在协同制造网络中的分配与调度,以多种成本之和的最小化为目标,提出了一种结合析取图的深度强化学习算法框架。根据输入的订单状态学习动作策略,将析取图的调度过程转化为一个多阶段的序列决策过程;利用基于析取图的状态空间,将协同制造订单的状态视为多通道图像输入网络,并依据状态转移的特点设计了包含订单选择规则和协同制造商指派规则的二维动作空间。根据问题的目标构造了关于成本的奖励函数,以指导智能体与环境交互,获取每个决策步最佳策略。实验结果表明,深度强化学习算法优于单一调度规则,与遗传算法相比较,在平均求解效果上表现更为优越。

关键词: 协同制造, 析取图, 深度强化学习, 调度, 马尔可夫决策过程