Maintenance decision-making using a continuous-state partially observable semi-Markov decision process

Due to the limitation of current condition monitoring technologies, the actual health state of an asset may not be revealed accurately by health inspections. A maintenance strategy ignoring this uncertainty of health state can cause additional costs or downtime. The partially observable Markov decision process (POMDP) is a commonly used approach to derive optimal maintenance strategies when the health inspection is imperfect. However, the existing applications of the POMDP to maintenance decision-making largely adopt the discrete time and state assumptions. The discrete-time assumption requires the health state transitions and maintenance activities only happen at discrete epochs, which cannot model the failure time accurately and is not cost-effective. The discrete health states, on the other hand, may not be elaborate enough to improve the effectiveness of maintenance. To address these limitations, this paper proposes a partially observable semi-Markov decision process (POSMDP) which is continuous in time and states. A Monte Carlo-based density projection method is adopted to convert the POSMDP to a complete observable semi-Markov decision process (SMDP). The converted SMDP is then solved by the policy iteration. At the end of this paper, a simulation study is performed to evaluate the performance of the POSMDP. The result shows that the maintenance strategy derived by the POSMDP is more cost-effective than another two maintenance strategies derived by approximate methods, when regular inspection intervals are adopted. The simulation study also shows that the maintenance cost can be further reduced by developing maintenance strategies with state-dependent maintenance intervals using the POSMDP.