DocumentCode :
933462
Title :
Properties of cross-entropy minimization
Author :
Shore, John E. ; Johnson, Rodney W.
Volume :
27
Issue :
4
fYear :
1981
fDate :
7/1/1981 12:00:00 AM
Firstpage :
472
Lastpage :
482
Abstract :
The principle of minimum cross-entropy (minimum directed divergence, minimum discrimination information) is a general method of inference about an unknown probability density when there exists a prior estimate of the density and new information in the form of constraints on expected values. Various fundamental properties of cross-entropy minimization are proven and collected in one place. Cross-entropy´s well-known properties as an information measure are extended and strengthened when one of the densities involved is the result of cross-entropy minimization. The interplay between properties of cross-entropy minimization as an inference procedure and properties of cross-entropy as an information measure is pointed out. Examples are included and general analytic and computational methods of finding minimum cross-entropy probability densities are discussed.
Keywords :
Entropy functions; Additive noise; Density measurement; Digital signal processing; Discrete transforms; Intersymbol interference; Mathematics; Minimization methods; Notice of Violation; Spectral analysis; Upper bound;
fLanguage :
English
Journal_Title :
Information Theory, IEEE Transactions on
Publisher :
ieee
ISSN :
0018-9448
Type :
jour
DOI :
10.1109/TIT.1981.1056373
Filename :
1056373
Link To Document :
بازگشت