A Stable Algorithm to Calculate Steady-State Probability & Frequency of a Markov System

This paper presents a stable method to calculate the steady-state probability and frequency of encountering a state in a Markov system. The method modifies the state transition matrix by reducing one state in each iteration until the Markov system reduces to a 2-state model. An algorithm for computer implementation is developed to calculate the probability and frequency of each state. Systems with up to 64 Markov states were solved using the approach. In each case, the method gave accurate results. The method does not involve any subtraction, so the method eliminates a source of subtractive cancellation errors. In contrast, the standard Gauss-elimination or Gauss-Jordan techniques can be affected by cancellation errors. A numerical example illustrates our approach.