A recursive Renyi´s entropy estimator

Author

Erdogmus, Deniz ; Principe, Jose C. ; Kim, Sung-Phil ; Sanchez, Justin C.

Author_Institution

Dept. of Electr. & Comput. Eng., Florida Univ., Gainesville, FL, USA

fYear

2002

fDate

2002

Firstpage

209

Lastpage

217

Abstract

Estimating the entropy of a sample set is required, in solving numerous learning scenarios involving information theoretic optimization criteria. A number of entropy estimators are available in the literature; however, these require a batch of samples to operate on in order to yield an estimate. We derive a recursive formula to estimate Renyi´s (1970) quadratic entropy on-line, using each new sample to update the entropy estimate to obtain more accurate results in stationary situations or to track the changing entropy of a signal in nonstationary situations.

Keywords

entropy; knowledge based systems; learning (artificial intelligence); recursive estimation; signal sampling; Renyi´s quadratic entropy; information theoretic optimization criteria; low-complexity learning rules; nonstationary signal; recursive Renyi´s entropy estimator; recursive formula; sample sequence; samples; Adaptive systems; Biomedical computing; Biomedical engineering; Digital communication; Entropy; Information theory; Neural networks; Recursive estimation; Stochastic processes; Yield estimation;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks for Signal Processing, 2002. Proceedings of the 2002 12th IEEE Workshop on

Print_ISBN

0-7803-7616-1

Type

conf

DOI

10.1109/NNSP.2002.1030032

Filename

1030032