Abstract: To explore the influence of deterioration effects on coupling relationship of maintenance and production scheduling,an integrated optimization was investigated for production scheduling and preventive maintenance in multi-state production systems.Based on preventive maintenance cost,production cost and finished job rewards,the integrated optimization problem was formulated as a Markov Decision Process (MDP) model of long-run expected average reward over finite-horizon.After analyzing and proving the existence of optimal stationary policy,an optimal equation was obtained for MDP model.To solve the difficulty that the traditional dynamic programming methods suffered from the curse of dimensionality and modeling,a model-free reinforcement learning algorithm was presented to solve the established MDP model on the basis of optimal equation.To evaluate the performance of reinforcement learning,a concise heuristic algorithm was proposed,and the experiments indicated that the reinforcement learning algorithm provide very effective solutions for the problem in comparison with the heuristic algorithm.A parameter sensitivity analysis was performed for the reinforcement learning algorithm,which provided the experiment reference for further design and improvement of the algorithm.

Key words: multi-state systems, integrated optimization, reinforcement learning, deterioration effects, production scheduling, preventive maintenance

摘要： 为了探索设备退化效应对维护和生产调度耦合关系的影响,对多态单机生产系统调度与预防性维护集成优化进行研究。基于预防性维护费用、工件加工成本以及工件完工回报值,建立了无限阶段平均期望报酬Markov决策模型,并分析和证明了该集成优化模型最优平稳策略的存在性,获得了该模型的最优方程。基于模型最优方程设计了一种无模型强化学习算法用于求解Markov决策模型,可有效解决传统动态规划算法在求解模型时所遭遇的维数灾和模型灾难题。为了评估该无模型强化学习的性能,基于无限阶段平均期望报酬设计了一种启发式求解算法,实验分析结果验证了所引入的无模型强化学习算法的有效性。最后,对该强化学习算法参数进行了敏感性分析,探索各参数对算法性能的影响,为算法设计及其性能提高提供了相应的实验依据。

关键词: 多态生产系统, 集成优化, 强化学习, 退化效应, 生产调度, 预测性维护