Entropy of the Mixture of Sources and Entropy Dimension

Suppose that we are given two sources S₁, S₂ which both send us information from the data space X . We assume that we lossy-code information coming from S₁ and S₂ with the same maximal error but with different alphabets P₁ and P₂, respectively. Consider a new source S which sends a signal produced by source S₁ with probability a₁ and by source S₂ with probability a₂=1-a₁ . We provide a simple greedy algorithm which constructs a coding alphabet P which encodes data from S with the same value of maximal error as single sources, such that the entropy h(S;P) satisfies: h(S;P) ≤ a₁h(S₁;P₁)+a₂h(S₂;P₂) +1. In the proof of the aforementioned formula, the basic role is played by a new equivalent definition of entropy based on measures instead of partitions. As a consequence, we decompose the entropy dimension of the mixture of sources in terms of the convex combination of the entropy dimensions of the single sources. In the case of probability measures in \BBR^N, this allows us to link the upper local dimension at point with the upper entropy dimension of a measure by an improved version of Young estimation.