Switching- and hedging- point policy for preventive maintenance with degrading machines: application to a two-machine line

Abstract

Maintenance and production are frequently managed as separate activities although they do interact. Disruptive events such as machine failures may find the company unready to repair the machine immediately leading to time waste. Preventive Maintenance may be carried out and maintenance time reduced to the effective task duration, in order to prevent time waste. Companies and researchers have been focusing on policies able to mitigate the impact of Preventive Maintenance on system availability, by exploiting the knowledge about degradation profiles in machines and the joint information from the machine state and the buffer level. In this work, the mathematical proof of the optimal threshold-based control policy for Preventive Maintenance with inventory cost, maintenance cost, backlog cost is provided. The control policy is defined in terms of buffer thresholds and dependency of the thresholds on the degradation condition. The optimal control policy is proved to include a combination of switching points and hedging points, where the first ones activate the Preventive Maintenance for a given condition and the latter ones control the production rate in order to minimize the surplus. An extensive experimental campaign analyzes the impact of system parameters such as the Maintenance duration on the cost function. The results show that there exists cases in which the optimal policy is dominated by the effect of the hedging points or the switching points, alternatively. Therefore, the proposed method is used to provide suggestions to the management for operative decisions, in order to choose the policy fitting best the system.

Appendix

We can derive Bellman’s equation:

$$\begin{aligned} J(S(t);t)= & {} \min _{u_c(s)} {\mathbb {E}}\left\{ \int _{t}^{t+\delta t}[g^+x^+(s) + g^-x^-(s)+ M\cdot [1 - \theta (s^u(s))] \right. \nonumber \\&\left. -\,A\cdot \theta (s^u(s))]ds + {\mathbb {E}}[J(S(t+\delta t);t + \delta t)] | S(t),t \right\} \end{aligned}$$

(47)

For $\delta t \rightarrow 0$ and developing $J(S(t+\delta t);t)$ to the first order:

$$\begin{aligned} J(S(t);t)= & {} \min _{u_c(t)} \left\{ [g^+x^+(t)+g^-x^-(t)+M\cdot [1 - \theta (s^u(t))] -A\cdot \theta (s^u(t))]dt \right. \nonumber \\&\left. +\,{\mathbb {E}}\left[ J(S(t);t) + \frac{\delta }{\delta t}J(S(t);t)dt + \frac{\delta }{\delta x}J(S(t);t)\delta x(t) + o(\delta t)\right] \right\} \end{aligned}$$

(48)

Since $\delta x(t) = \frac{dx(t)}{dt} \delta t =[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\delta t$:

$$\begin{aligned} J(S(t);t)= & {} \min _{u_c(t)} \left\{ [g^+x^+(t)+g^-x^-(t)+M\cdot [1 - \theta (s^u(t))] -A\cdot \theta (s^u(t))]dt \right. \nonumber \\&+\,{\mathbb {E}}\left[ J(S(t);t) + \frac{\delta }{\delta t}J(S(t);t)dt \right. \nonumber \\&\left. \left. +\,\frac{\delta }{\delta x}J(S(t);t)[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\delta t + o(\delta t)\right] \right\} \end{aligned}$$

(49)

Taking the expectations over all the possible states:

$$\begin{aligned} J(S(t);t)= & {} \min _{u_c(t)} \left\{ g^+x^+(t)dt+g^-x^-(t)dt+M\cdot [1 - \theta (s^u(t))]dt -A\cdot \theta (s^u(t))dt \right. \nonumber \\&+ J(S(t);t) + \sum _i J(S_i(t);t)\lambda ^u_{s^u(t),i}\delta t \nonumber \\&+ \frac{\delta }{\delta t}J(S(t);t) + \sum _i \frac{\delta }{\delta t} J(S_i(t);t)\lambda ^u_{s^u(t),i}\delta t \delta t \nonumber \\&+ \frac{\delta }{\delta x}J(S(t);t)[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\delta t \nonumber \\&\left. + \sum _i J(S_i(t);t) [\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\lambda ^u_{s^u(t),i}\delta t \delta t +o(\delta t)\right\} \end{aligned}$$

(50)

Eliminating the infinitesimal terms of higher order:

$$\begin{aligned} J(S(t);t)= & {} \min _{u_c(t)} \left\{ g^+x^+(t)dt+g^-x^-(t)dt+M\cdot [1 - \theta (s^u(t))]dt -A\cdot \theta (s^u(t))dt \right. \nonumber \\&+ J(S(t);t) + \sum _i J(S_i(t);t)\lambda ^u_{s^u(t),i}\delta t + \frac{\delta }{\delta t}J(S(t);t)\nonumber \\&\left. +\,\frac{\delta }{\delta x}J(S(t);t)[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\delta t\right\} \end{aligned}$$

(51)

Dividing by $\delta t$, simplifying and taking out from the minimum the terms that do not depend on $u_c(t)$:

$$\begin{aligned} - \frac{\delta }{\delta t}J(S(t);t)= & {} \min _{u_c(t)} \left\{ g^+x^+(t)+g^-x^-(t)+M\cdot [1 - \theta (s^u(t))] -A\cdot \theta (s^u(t))\right. \nonumber \\&+ \sum _i J(S_i(t);t)\lambda ^u_{s^u(t),i} \nonumber \\&\left. +\,\frac{\delta }{\delta x}J(S(t);t)[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\right\} \end{aligned}$$

(52)

Now, as far as the following conditions are true:

• the system dynamics does not depend directly on t;

• a steady state distribution exists for $s^u(t), s^d(t)$

• it is possible to find a control policy that keeps x bounded.

Then the optimal control policy does not depend on t and the probability distribution of $u_c(t)$ is approximately constant in t. Therefore $J(S(t);t)\approx J(S,t)$. It is possible to define J(S, t) with the following expression (see Gershwin, 1994):

$$\begin{aligned} J(S(t);t) = J(x(t);s^u(t);s^d(t);t)\approx J(x,s^u,s^d,t) = J^*(t-t) + W(x,s^u,s^d) \end{aligned}$$

(53)

with:

$$\begin{aligned} \frac{\delta J}{\delta t}\approx & {} -J^*\nonumber \\ \frac{\delta J}{\delta x}\approx & {} \frac{\delta W(x,s^u,s^d)}{\delta x} \end{aligned}$$

(54)

Therefore, the Bellman’s equation becomes:

$$\begin{aligned} - \frac{\delta }{\delta t}J(S(t);t)= & {} \min _{u_c(t)} \left\{ g^+x^+(t)+g^-x^-(t)+M\cdot [1 - \theta (s^u(t))] -A\cdot \theta (s^u(t))+\right. \nonumber \\&+ \sum _i J(S_i(t);t)\lambda ^u_{s^u(t),i} \nonumber \\&\left. + \frac{\delta }{\delta x}J(S(t);t)[\theta (s^u(t))\cdot \mu (t)_u - \mu (t)_d]\right\} \end{aligned}$$

(55)

$$\begin{aligned} J^*= & {} \min _{u_c)} \left\{ g^+x^+ +g^-x^- +M\cdot [1 - \theta (s^u)] - A \cdot \theta (s^u)\right. \nonumber \\&+ \sum _i [J^*(T-t) + W(x,i,s^d)]\lambda ^u_{s^u,i} \nonumber \\&\left. +\,\frac{\delta W(x,s^u,s^d)}{\delta x}\left[ \theta (s^u)\cdot \mu ^u - \mu ^d \right] \right\} \end{aligned}$$

(56)

$$\begin{aligned} J^*= & {} \min _{u_c} \left\{ g^+x^+ +g^-x^- + M\cdot [1 - \theta (s^u)] - A \cdot \theta (s^u)+\right. \nonumber \\&+ J^*(T-t) \sum _i \lambda ^u_{s^u,i} + \sum _i W(x,i,s^d) \lambda ^u_{s^u,i} \nonumber \\&\left. +\,\frac{\delta W(x,s^u,s^d)}{\delta x}\left[ \theta (s^u)\cdot \mu ^u - \mu ^d \right] \right\} \end{aligned}$$

(57)

Since $\sum _i \lambda ^u_{s^u,i} = 0$ and $\sum _j \lambda ^d_{s^d,j} = 0$ by definition, we have:

$$\begin{aligned} J^*= & {} \min _{u_c} \{g^+x^+ +g^-x^- + M\cdot [1 - \theta (s^u)] - A \cdot \theta (s^u)\nonumber \\&+ \sum _i W(x,i,s^d) \lambda ^u_{s^u,i} \nonumber \\&+\,\frac{\delta W(x,s^u,s^d)}{\delta x}\left[ \theta (s^u)\cdot \mu ^u - \mu ^d \right] \} \end{aligned}$$

(58)