The overall performance of the Shannon code

It is well-known that the redundancy of the Shannon code lies in the interval (0, 1). In order to study the overall performance of the Shannon code, we consider its redundancy as a random variable on the set of sources with n symbols, i.e., R_sh(n), and examine its statistical parameters. It is shown that the mean of R_sh(n) gets close to 0.5 for sources with large alphabet size n. Moreover, we observe that its variance tends to zero as n increases. Briefly speaking, for almost all sources with a large alphabet size, the redundancy of the Shannon code is almost 0.5 bits.