Taylor Expansion for the Entropy Rate of Hidden Markov Chains

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chainʹs and channelʹs parameters. Then, we try to obtain an estimate for the entropy rate of hidden Markov chain by matrix algebra and its spectral representation. To do so, we use the Taylor expansion, and calculate some estimates for the first and the second terms, for the entropy rate of the hidden Markov process and its binary version, respectively. For small ? (channelʹs parameter), the entropy rate has ?(?^2 ), as a maximum error, when it is calculated by the first term of Taylor expansion and it has ?(?^3 ), as a maximum error, when it is calculated by the second term.