Title :
Estimation of the information by an adaptive partitioning of the observation space
Author :
Darbellay, Georges A. ; Vajda, Igor
Author_Institution :
Inst. of Inf. Theory & Autom., Czechoslovak Acad. of Sci., Prague, Czech Republic
fDate :
5/1/1999 12:00:00 AM
Abstract :
We demonstrate that it is possible to approximate the mutual information arbitrarily closely in probability by calculating the relative frequencies on appropriate partitions and achieving conditional independence on the rectangles of which the partitions are made. Empirical results, including a comparison with maximum-likelihood estimators, are presented
Keywords :
adaptive estimation; approximation theory; frequency estimation; information theory; nonparametric statistics; probability; adaptive partitioning; conditional independence; information estimation; maximum-likelihood estimators; mutual information approximation; nonparametric estimator; observation space; partitions; probability; random variables; rectangles; relative frequencies; Density measurement; Entropy; Frequency; Histograms; Information theory; Maximum likelihood estimation; Multidimensional systems; Mutual information; Probability; Random variables;
Journal_Title :
Information Theory, IEEE Transactions on
