Optimal maintenance policy for systems subject to random shocks: a semi-Markov decision process approach

2.1 Notation

The notation used throughout the model is summarized in Table 1.

2.2 Assumptions

The system under consideration is a single-component system that undergoes continuous degradation and random shocks. The key assumptions of the model are as follows:

1. Random shocks occur according to a Poisson process $$ N(t) $$ with rate λ;

2. The degradation increment over an interval of length t follows a Gamma distribution with shape parameter $$ \alpha t $$ and rate parameter β;

3. The degradation process and the shock process are independent of each other;

4. A system fails when the accumulated number of shocks reaches the failure threshold M or the degradation level reaches or exceeds the threshold L, whichever occurs first;

5. Failures are non-self-announcing; the failure status, accumulated shock count, and degradation level can only be revealed through inspections.

Remark 1. The independence assumption is adopted to establish a tractable baseline model for the joint optimization of inspection scheduling and replacement decisions. It is suitable for engineering systems in which internal degradation is mainly caused by gradual physical deterioration, such as wear, corrosion, fatigue, or aging, while external shocks are caused by random environmental or operational disturbances. Under this assumption, the degradation-induced failure event and the shock-induced failure event can be modeled as two competing failure mechanisms with explicit transition probabilities. Nevertheless, this assumption may not hold for systems with strong degradation-shock dependence. For example, shocks may accelerate the degradation process, and a high degradation level may increase the system's vulnerability to shocks. Such dependence is beyond the scope of the present study and is left for future research.

Remark 2. The present model assumes that inspections are perfect, meaning that the accumulated shock state and the degradation level can be accurately observed at each inspection epoch. This assumption is commonly used as a baseline setting for inspection-based maintenance optimization because it allows the state-dependent inspection interval and replacement decision to be explicitly characterized. In practice, however, inspections may be subject to measurement errors, missed detections, or delayed fault identification. Incorporating such inspection uncertainty would require the observed state to be replaced by a belief state or an estimated degradation distribution, leading to a partially observable decision model. This extension is meaningful but beyond the scope of the present study.

Define the system state as a two-dimensional vector $$ (m, x) $$, where m represents the accumulated number of shocks and x represents the degradation level, with $$ m = 0, 1, \dots, M $$ and $$ 0\leq x<\infty $$. Following the standard Gamma-process degradation formulation in maintenance modeling^[31], the probability density function of the degradation increment during an interval of length t is given by

Accordingly, the degradation-induced failure probability within an interval of length t, starting from degradation level x, can be expressed by the upper tail probability of the Gamma distribution^[31]:

2.3 Transition probability

For a Poisson shock process, the inter-arrival times are independent and identically distributed exponential random variables, and the arrival time of the l-th shock follows an Erlang distribution^[32]. Therefore, for $$ l=1, 2, \ldots, M $$, let $$ h_l(t) $$ denote the probability density function of the arrival time of the l-th shock with parameter λ, given by

The corresponding cumulative distribution function is^[32]

Next, let $$ P_{m, m'}(t) $$ represent the probability that the accumulated shock state transitions from m to $$ m' $$ during a time interval of length t, where $$ 0\leq m\leq m'\leq M $$ and $$ t>0 $$. Since state M represents the shock-failure state, the transition to M aggregates all sample paths in which the accumulated number of shocks reaches or exceeds the threshold M during the interval. Thus, $$ P_{m, m'}(t) $$ can be expressed as shown in Eq. (5). The interpretation of these probabilities is as follows:

1. If $$ m = m'\leq M - 1 $$, no shock occurs within the time interval, i.e., the arrival time for the first shock is longer than t;

2. If $$ m <m' \leq M - 1 $$, the arrival time for $$ (m' - m) $$ successive shocks is shorter than t, while the arrival time for $$ (m' - m) + 1 $$ successive shocks exceeds t;

3. If $$ m<m' = M $$, the arrival time for $$ (M - m) $$ successive shocks is shorter than t;

2.4 Mean downtime duration

Let $$ F_{m, x}(t) $$ denote the probability that the system fails within a time interval of length t, starting from the initial state $$ (m, x) $$, where $$ m=0, 1, \ldots, {M}-1 $$ and $$ 0\leq x<\infty $$. This probability is

Let $$ \tau_{m, x} $$ denote the failure time measured from the start of an inspection interval. If failure occurs before the next inspection at time t, the system remains down for $$ t-\tau_{m, x} $$. Hence, using the tail-integral identity for the nonnegative random variable $$ (t-\tau_{m, x})^{+} $$, the expected downtime during the interval is

3.1 Maintenance actions and renewal equations

In this section, we formulate the average-cost optimal maintenance problem. Two types of maintenance actions are considered: non-periodic inspection, denoted by $$ (\mathrm{NI}, t) $$, and immediate replacement, denoted by $$ (\mathrm{IR}) $$. Let π denote a maintenance policy, where the action prescribed at state $$ (m, x) $$ is denoted by

Both actions are admissible at any non-failed system state $$ (m, x) $$. Specifically,

1. If action $$ (\mathrm{NI}, t) $$ is selected, the system is inspected after a non-periodic inspection interval of length t.

2. If action $$ (\mathrm{IR}) $$ is selected, the system is replaced immediately and restored to the as-good-as-new state (0, 0). Remark 3. In this study, we focus on non-periodic inspection and immediate replacement actions in order to clearly characterize the joint optimization of inspection timing and replacement decisions. Imperfect or partial maintenance actions, such as minor repair or degradation reduction, are not included in the current action set. If such actions are considered, the action space should be expanded and the state transition probabilities should be modified to describe the post-maintenance degradation state and shock vulnerability. This extension would enrich the maintenance decision structure, but it would also require additional assumptions on the effectiveness and cost of imperfect maintenance.

Under a given maintenance policy π, let $$ \mathrm{C}_{\pi}(m, x) $$ and $$ \mathrm{T}_{\pi}(m, x) $$ denote the expected cost and the expected elapsed time, respectively, from state $$ (m, x) $$ until the system reaches the renewal state (0, 0). Accordingly, $$ \mathrm{C}_{\pi}(0, 0) $$ and $$ \mathrm{T}_{\pi}(0, 0) $$ represent the expected renewal cost and the expected renewal time of one renewal cycle, respectively.

The recursive expressions for $$ \mathrm{C}_{\pi}(m, x) $$ and $$ \mathrm{T}_{\pi}(m, x) $$ depend on the action selected at state $$ (m, x) $$. Specifically, the expressions are derived based on whether the policy prescribes a non-periodic inspection $$ (\mathrm{NI}, t) $$ or an immediate replacement $$ (\mathrm{IR}) $$ at that state.

1. If policy π prescribes action $$ (\mathrm{NI}, t) $$ at state $$ (m, x) $$, the expected renewal cost consists of the inspection cost, the expected downtime cost during the inspection interval, and the expected cost incurred after the inspection interval until the next renewal. Similarly, the expected renewal time consists of the inspection interval length and the expected remaining time to renewal. Conditioning on the shock state $$ m' $$ and the degradation increment y at the next inspection epoch gives

2. If policy π prescribes action $$ (\mathrm{IR}) $$ at state $$ (m, x) $$, the system is replaced immediately and a renewal occurs at once. Hence,

where $$ C_p $$ is the preventive replacement cost and $$ C_c $$ is the corrective replacement cost.

3.2 Excess-cost Bellman optimality equation

To derive the optimal maintenance policy, we follow the standard average-cost Markov decision process framework and transform the average-cost problem into a parametric excess-cost problem^[33]. Let g denote a candidate average cost rate. For each non-failed state $$ (m, x) $$, define $$ V^{\mathrm{IR}}_{g}(m, x) $$ and $$ V^{\mathrm{NI}}_{g}(m, x;t) $$ as the g-adjusted expected costs associated with immediate replacement and non-periodic inspection, respectively.

If immediate replacement $$ (\mathrm{IR}) $$ is selected at state $$ (m, x) $$, the system is replaced immediately and restored to the as-good-as-new state. Therefore, the g-adjusted cost of immediate replacement is

If non-periodic inspection $$ (\mathrm{NI}, t) $$ is selected at state $$ (m, x) $$, the system incurs the inspection cost, the expected downtime cost during the inspection interval, and the time-adjusted term $$ -gt $$. After the inspection interval, the system moves to a new state $$ (m', x+y) $$, from which the optimal policy is followed. Conditioning on this next state gives

For a fixed candidate cost rate g, the optimal inspection interval associated with the non-periodic inspection action is obtained by

and the minimized g-adjusted cost of non-periodic inspection is

For a fixed candidate average cost rate g, the original average-cost problem is converted into an excess-cost minimization problem. At each decision epoch, the decision maker needs to compare only the g-adjusted costs of the two admissible action types: immediate replacement and non-periodic inspection. The former terminates the current renewal cycle immediately, whereas the latter incurs the inspection cost, the expected downtime cost, the time-adjusted term $$ -gt $$, and the future optimal excess cost after the next inspection epoch. Therefore, by the dynamic programming optimality principle for average-cost decision processes^[33], the optimal excess-cost function satisfies the Bellman optimality equation

This equation is valid because the next decision depends on the past history only through the observed state $$ (m, x) $$ at the inspection epoch. Hence, the state $$ (m, x) $$ is sufficient for dynamic programming, and the optimal action can be selected by comparing the two action values in Eq. (14). Specifically, immediate replacement is optimal if

Otherwise, non-periodic inspection is optimal, and the corresponding inspection interval is $$ t_g^*(m, x) $$.

After discretization, the transition structure permits a backward recursion because both the accumulated shock count and the degradation level are nondecreasing. After setting the boundary values for the failed states, the non-failed grid states are evaluated from larger to smaller values of m and x. The optimal maintenance policy is then obtained by comparing the two action values at each state.

3.3 Determination of the optimal average cost rate

The value of g in Eq. (14) is only a candidate average cost rate. Therefore, solving Eq. (14) for a given g yields the policy that is optimal with respect to this candidate benchmark, but it does not directly determine the optimal average cost rate. To identify the true optimal average cost rate, we examine the optimal excess cost at the renewal state (0, 0).

For any admissible maintenance policy π, the expected excess cost of one renewal cycle under the candidate cost rate g is

Equivalently,

Since $$ \mathrm{T}_{\pi}(0, 0)>0 $$, the sign of $$ V_g^{\pi}(0, 0) $$ indicates whether the average cost rate of policy π is larger or smaller than the candidate value g.

After optimizing over all admissible policies, the optimal excess cost at the renewal state equals

Thus, $$ V_g^*(0, 0) $$ provides a criterion for determining whether the candidate cost rate g is below or above the optimal average cost rate. If $$ V_g^*(0, 0)>0 $$, then every admissible policy has an average cost rate larger than g, which implies that g is smaller than the optimal average cost rate. If $$ V_g^*(0, 0)<0 $$, then there exists at least one policy whose average cost rate is smaller than g, which implies that g is larger than the optimal average cost rate.

Consequently, the optimal average cost rate $$ g^* $$ is the unique root of $$ V_g^*(0, 0)=0 $$. Equivalently, it satisfies

At this root,

The optimal maintenance policy is then obtained from Eq. (14) by setting $$ g=g^* $$.

3.4 Numerical implementation and convergence check

The continuous degradation state is discretized on a uniform grid over $$ [0, L] $$ with grid size $$ \Delta x=0.1 $$. The value function is evaluated at the grid points, and the integrals with respect to the Gamma degradation increment are computed numerically on the same degradation grid. For the non-periodic inspection action, the inspection interval is searched over a one-dimensional grid with step size $$ \Delta t=0.01 $$. If the next degradation level $$ x+y $$ does not exactly fall on a grid point, linear interpolation is used to approximate the corresponding value function.

For a fixed candidate average cost rate g, the discretized Bellman equation is solved recursively over the state space. Boundary values are first assigned to failed states with $$ m=M $$ or $$ x\geq L $$. Because future states have shock counts and degradation levels no smaller than their current values, the non-failed grid states are evaluated backward from larger to smaller m and x. After solving the Bellman equation for a given g, the optimal average cost rate is obtained by applying bisection to the root equation $$ V_g^*(0, 0)=0 $$. The bisection search is terminated when either $$ |V_g^*(0, 0)|\leq 10^{-6} $$ or the length of the bisection interval for g is less than $$ 10^{-6} $$.

4.1 Results for the proposed policy

Based on the baseline parameter setting, the optimal average cost rate is obtained by solving the root equation $$ V_g^*(0, 0)=0 $$, yielding $$ \mathrm{ACR}=g^*=106.60 $$. Substituting this value into the Bellman optimality equation gives the state-dependent maintenance policy and the corresponding excess-cost function, as shown in Figure 1. Specifically, Figure 1 presents the optimal inspection interval $$ t_{g^*}^*(m, x) $$ and the optimal excess-cost function $$ V_{g^*}^*(m, x) $$ under different accumulated shock states.

Figure 1. Optimal inspection intervals (left panel) and excess-cost function (right panel) for different operating states. The shock-failure state m=M=5 is excluded because the inspection interval is not optimized after failure.

The left panel of Figure 1 shows that the optimal inspection interval is strongly state-dependent. For a fixed accumulated shock state m, the inspection interval generally decreases as the degradation level x increases, because the system approaches the degradation failure threshold and requires more frequent inspections to reduce the expected downtime caused by non-self-announcing failures. Similarly, for a fixed degradation level, the inspection interval tends to decrease as m increases, because the system approaches the shock failure threshold. Thus, the policy becomes more conservative as the system deteriorates. When both m and x are small, the system is relatively healthy, allowing longer inspection intervals, which helps avoid unnecessary inspection costs. By contrast, when the system approaches either the shock failure threshold or the degradation failure threshold, the inspection interval becomes shorter, and immediate replacement is selected in states where no inspection interval is plotted. Therefore, the proposed policy adaptively balances inspection cost, downtime cost, and replacement cost according to the observed system state. In particular, because the baseline threshold is $$ M=5 $$, the state $$ m=5 $$ corresponds to the shock-failure state $$ m=M $$. According to the model definition, this state is not an operational state for which an inspection interval can be optimized. Therefore, the corresponding case has been excluded from the optimal inspection interval analysis in Figure 1. The right panel of Figure 1 presents the excess-cost function $$ V_{g^*}^*(m, x) $$ for the remaining operational states. The value function increases as the system state deteriorates, reflecting the higher expected cost-to-renewal from more severe states. In particular, states with larger m generally have higher excess costs because they are closer to the shock-induced failure boundary. The flat curve for $$ m=4 $$, as well as the common plateau reached by the other curves near the degradation boundary, corresponds to non-failed states where immediate preventive replacement is optimal. In these states, $$ m<M $$ and $$ x<L $$, so the excess cost equals $$ C_p=2000 $$ rather than the corrective replacement cost $$ C_c $$; corrective replacement applies only after failure ($$ m=M $$ or $$ x\geq L $$). These results confirm that the proposed dynamic policy captures the joint effect of cumulative shocks and degradation on maintenance decisions.

Remark 4. To examine the influence of discretization error, the computation was repeated using different grid resolutions for both the degradation state and the inspection interval. The relative difference is computed with respect to the baseline grid, i.e., $$ |\mathrm{ACR}-106.60|/106.60\; \times\; 100\% $$. As shown in Table 3, the coarse grid with $$ \Delta x=0.20 $$ and $$ \Delta t=0.02 $$ gives an average cost rate of 108.56, a 1.84% deviation from the baseline result. When a finer grid with $$ \Delta x=0.05 $$ and $$ \Delta t=0.005 $$ is used, the average cost rate becomes 106.54, differing from the baseline result by only 0.06%. These results indicate that the reported average cost rate is stable with respect to the discretization step and that the baseline discretization is sufficiently accurate for the numerical analysis.

4.2 Comparison with a benchmark policy

To evaluate the effectiveness of the proposed state-dependent non-periodic inspection policy, we compare it with a benchmark periodic inspection and replacement policy^[34,35], denoted by Policy B. Under Policy B, inspections are performed at a fixed interval T, and preventive replacement is conducted when the observed degradation level reaches or exceeds a predetermined threshold $$ L_p $$. Therefore, Policy B is characterized by two decision variables, the periodic inspection interval T and the preventive replacement threshold $$ L_p $$. For a fair comparison, the optimal values of T and $$ L_p $$ are obtained by minimizing the long-run average cost rate under the same system parameters listed in Table 2.

Table 4 reports the optimal benchmark policy and its performance. The optimal periodic inspection interval is $$ T^*=5.00 $$, and the corresponding preventive replacement threshold is $$ L_p^*=8.70 $$. Under this benchmark policy, the minimum average cost rate is 123.46. In contrast, the proposed policy yields an average cost rate of 106.60 under the baseline setting. Let $$ \mathscr{R} $$ denote the percentage reduction in the average cost rate achieved by the proposed policy compared with the benchmark policy, defined as

where $$ \mathrm{ACR}_{\mathrm{A}} $$ and $$ \mathrm{ACR}_{\mathrm{B}} $$ are the average cost rates of the proposed policy and the benchmark policy, respectively.

As shown in Table 4, the proposed policy reduces the average cost rate by 13.66% compared with the optimized benchmark policy. This improvement is mainly due to the state-dependent structure of the proposed policy. Unlike the benchmark policy, which uses a fixed inspection interval regardless of the current system condition, the proposed policy dynamically adjusts the next inspection interval according to both the accumulated number of shocks m and the degradation level x. When the system is close to either the shock failure threshold or the degradation failure threshold, the proposed policy shortens the inspection interval to reduce the risk of undetected failure and the associated downtime cost. Conversely, when the system is in a relatively healthy state, longer inspection intervals can be adopted to avoid unnecessary inspection costs. Therefore, the proposed policy achieves a better balance among inspection cost, downtime cost, and replacement cost.

4.3 Sensitivity analysis

To examine the robustness of the proposed policy and to understand the effects of different model parameters, a sensitivity analysis is conducted. Starting from the baseline parameter setting in Table 2, each parameter is varied individually while all the other parameters are kept unchanged. Specifically, each parameter is multiplied by 0.5, 0.75, 1.0, 1.25, and 1.5, respectively. For the shock threshold M, the adjusted value is rounded upward to the nearest integer.

Table 5 reports the average cost rate of the proposed policy under each parameter setting. The value in parentheses represents the percentage reduction in average cost rate compared with the optimized benchmark policy under the same parameter setting. Several observations can be obtained from Table 5. First, the proposed policy consistently outperforms the benchmark policy for all parameter settings, as all reported cost-rate reductions are positive. This confirms the robustness of the proposed state-dependent policy. Second, the optimal average cost rate increases with the cost parameters $$ C_c $$, $$ C_d $$, $$ C_p $$, and $$ C_i $$. Among them, $$ C_p $$ has the most direct impact on the average cost rate, increasing from 65.11 to 143.70 as $$ C_p $$ varies from 50% to 150% of its baseline value. This is expected because preventive replacement is one of the major cost components in the maintenance policy. The effect of $$ C_d $$ is relatively moderate in terms of the absolute average cost rate, but the cost reduction ratio increases as $$ C_d $$ becomes larger, indicating that the proposed policy is particularly beneficial when downtime is costly. The stochastic parameters also have clear effects on the average cost rate. When M increases, the system can tolerate more random shocks before failure, and thus the average cost rate decreases. In contrast, a larger shock arrival rate λ leads to a higher risk of shock-induced failure and hence a larger average cost rate. For the degradation process, increasing α accelerates degradation and results in a higher average cost rate. Under the Gamma density parameterization adopted in this paper, β is the rate parameter; hence, a larger β reduces the mean degradation increment. Therefore, the average cost rate decreases as β increases. These trends are consistent with the physical interpretation of the shock and degradation processes.

To further illustrate how the optimal policy changes with parameter variations, Figures 2 and 3 plot the optimal inspection interval $$ t_g^*(m, x) $$ under different parameter settings. Figure 2 focuses on the cost parameters, while Figure 3 focuses on the shock and degradation parameters. In each subplot, the horizontal axis is the degradation level x, and each curve corresponds to a different accumulated shock state m. If immediate replacement is optimal at a state, the corresponding inspection interval is zero and is not plotted. As shown in Figures 2 and 3, the optimal inspection interval generally decreases as the degradation level x increases. This is because a larger degradation level means that the system is closer to the degradation failure threshold L, so more frequent inspections are required to reduce the expected downtime caused by non-self-announcing failures. For a fixed degradation level, the inspection interval also tends to be shorter when the accumulated number of shocks m is larger, since the system is then closer to the shock failure threshold M. Therefore, the optimal policy exhibits a clear state-dependent structure: inspections are scheduled more frequently when the system condition becomes worse.

Figure 2. Sensitivity of the optimal inspection interval with respect to cost parameters. Different curves represent different accumulated shock states m and are distinguished by line styles and markers.

Figure 3. Sensitivity of the optimal inspection interval with respect to shock and degradation parameters. Different curves represent different accumulated shock states m and are distinguished by line styles and markers.

Figure 2 shows that the cost parameters affect the inspection policy in different ways. As $$ C_c $$ increases, the optimal inspection intervals tend to decrease because corrective replacement becomes more expensive and the policy becomes more conservative. A similar effect is observed when $$ C_d $$ increases, because a higher downtime cost rate increases the penalty of hidden failures during an inspection interval. By contrast, increasing $$ C_p $$ tends to increase the inspection intervals in several states, since preventive replacement becomes more expensive and the policy is more inclined to postpone replacement. The inspection cost $$ C_i $$ also has a positive effect on the inspection interval: when each inspection becomes more costly, the policy avoids excessively frequent inspections and chooses longer intervals.

Figure 3 further confirms the influence of the shock and degradation parameters. When M increases, more shock states are allowed before failure, and the policy includes more curves corresponding to larger values of m. For smaller m, the system is relatively far from the shock failure boundary, and longer inspection intervals are permitted. As λ increases, shocks arrive more frequently, and the optimal inspection intervals become shorter. For the degradation parameters, a larger α leads to faster degradation and thus shorter inspection intervals. In contrast, a larger β slows down the degradation process under the rate-parameter formulation, so the optimal inspection intervals become longer.

Overall, the sensitivity analysis shows that the proposed policy can adaptively respond to changes in both economic and stochastic parameters. The policy schedules shorter inspection intervals when failure risk or failure-related costs increase, and longer inspection intervals when inspections or preventive replacements become more expensive or when the degradation process becomes less severe. These results demonstrate the flexibility of the proposed non-periodic inspection policy and explain its consistent advantage over the benchmark periodic policy.