The entropy of ordered sequences and order statistics

Author

Wong, Kon M. ; Chen, Shuang

Author_Institution

Dept. of Electr. & Comput. Eng., McMaster Univ., Hamilton, Ont., Canada

Volume

36

Issue

2

fYear

1990

fDate

3/1/1990 12:00:00 AM

Firstpage

276

Lastpage

284

Abstract

The entropy of a sequence of random variables under order restrictions is examined. A theorem that shows the amount of entropy reduction when the sequence is ordered is presented. Upper and lower bounds to the entropy reduction and conditions under which they are achieved are derived. Some interesting properties of the entropy of the individual order statistics are also presented. It is shown that the difference between the average entropy of the individual order statistics and the entropy of a member of the original independent identically distributed (IID) population is a constant, regardless of the original distribution. Finally, the entropies of the individual order statistics are found to be symmetric about the median when the probability density function (PDF) of the original IID sequence is symmetric about its mean

Keywords

entropy; information theory; statistics; IID sequence; entropy; entropy reduction; independent identically distributed; lower bounds; order statistics; ordered sequences; probability density function; random variables; upper bounds; Entropy; Filters; Image processing; Noise robustness; Probability density function; Random variables; Signal processing; Speech processing; Statistical distributions; Statistics;

fLanguage

English

Journal_Title

Information Theory, IEEE Transactions on

Publisher

ieee

ISSN

0018-9448

Type

jour

DOI

10.1109/18.52473

Filename

52473