This paper discusses the predictive maintenance (PM) problem of a single equipment system. It is assumed that the equipment has deteriorating quality states as it operates, resulting in multiple yield levels represented as system observation states. We cast the equipment deterioration as discrete-state and continuous-time semi-Markov decision process (SMDP) model and solve the SMDP problem in reinforcement learning (RL) framework using the strategy-based method. In doing so, the goal is to maximize the system average reward rate (SARR) and generate the optimal maintenance strategy for given observation states. Further, the PM time is capable of being produced by a simulation method. In order to prove the advantage of our proposed method, we introduce the standard sequential preventive maintenance algorithm with unequal time interval. Our proposed method is compared with the sequential preventive maintenance algorithm in a test objective of SARR, and the results tell us that our proposed method can outperform the sequential preventive maintenance algorithm. In the end, the sensitivity analysis of some parameters on the PM time is given.

中文翻译：

---

具有多个质量状态的设备的预测性维护和灵敏度分析

本文讨论了单个设备系统的预测性维护（PM）问题。假定设备在运行时具有恶化的质量状态，从而导致多个产量水平表示为系统观察状态。我们将设备退化情况作为离散状态和连续时间的半马尔可夫决策过程（SMDP）模型，并使用基于策略的方法在强化学习（RL）框架中解决了SMDP问题。这样做的目的是使系统平均奖励率（SARR）最大化，并针对给定的观察状态生成最佳维护策略。此外，PM时间能够通过模拟方法产生。为了证明我们提出的方法的优势，我们引入了具有不相等时间间隔的标准顺序预防性维护算法。在SARR的测试目标中，将本文提出的方法与顺序预防性维护算法进行了比较，结果表明，本文提出的方法的性能优于顺序预防性维护算法。最后，给出了一些参数对PM时间的敏感性分析。