DocumentCode :
997975
Title :
The uniform distribution as a universal prior
Author :
Shulman, Nadav ; Feder, Meir
Author_Institution :
Dept. of Electr. Eng.-Syst., Tel-Aviv Univ., Israel
Volume :
50
Issue :
6
fYear :
2004
fDate :
6/1/2004 12:00:00 AM
Firstpage :
1356
Lastpage :
1362
Abstract :
In this correspondence, we discuss the properties of the uniform prior as a universal prior, i.e., a prior that induces a mutual information that is simultaneously close to the capacity for all channels. We determine bounds on the amount of the mutual information loss in using the uniform prior instead of the capacity-achieving prior. Specifically, for the class of binary input channels with any output alphabet, we show that the Z-channel has the minimal mutual information with uniform prior, out of all channels with a given capacity. From this, we conclude that the degradation of the mutual information with respect to the capacity is at most 0.011 bit, and as was shown previously, at most 6%. A related result is that the capacity-achieving prior, for any channel, is not far from uniform. Some of these results are extended to channels with nonbinary input.
Keywords :
channel capacity; information theory; Z-channel; binary input channel; channel capacity; mutual information degradation; nonbinary input; uniform distribution; uniform prior; universal prior; Channel capacity; Degradation; Linear code; Maintenance; Monitoring; Mutual information; Transmitters; Upper bound; $Z$-channel; Uniform distribution; universal prior;
fLanguage :
English
Journal_Title :
Information Theory, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9448
Type :
jour
DOI :
10.1109/TIT.2004.828152
Filename :
1302318
Link To Document :
بازگشت