Redundancy and optimality of codes for infinite-entropy sources

Redundancy and optimality are standard concepts for prefix codes for finite-entropy sources, but they can be extended to codes for (discrete) infinite-entropy sources as well. Here three redundancy definitions and two optimality definitions are considered that accomplish this. The redundancy definitions turn out to be equivalent, and they are also equivalent to the ´expected codeword length minus entropy´ definition when the source entropy is finite. An analogous statement holds for the optimality definitions. A theorem on the redundancy of finite sources is also presented; it provides a simple bound on the difference between the redundancies of optimal codes for two sources that contain an identical subset of symbol probabilities.