Consistent Density Function Estimation with Multilayer Perceptrons

A consistent density function estimator is presented. Whether an estimator is consistent or not is critical when the desired solution is not known. Without determining consistency it is not possible to know if the solution generated by an algorithm is close to the true solution or not. A combination of performance index, MLP architecture, training algorithm, and statistical learning theory concepts is used to produce consistent one dimensional density function estimations. The performance index and the MLP architecture are designed using information theoretical and algorithmic considerations, whereas the consistency of the solution is determined from the behavior exhibited by the estimator throughout the training process. The training algorithm is designed to highlight behavior that has been proven to exist in other learning problems with the help of statistical learning theory. The algorithm is tested with examples in order to determine the extent of its usefulness and to study its limitations.