Adaptive Probability Distribution Estimation Based upon Maximum Entropy

Our ad-hoc adaptive estimation procedure for the probability distribution of a continuous random variable is based upon the Shannon-Jaynes maximum entropy concept and uses regression techniques or the Kullback-Leibler Divergence measure of information variation to select the appropriate functions for fitting a regular exponential family distribution to the data. This parametric estimation technique uses the data to select the probability distribution and estimate the parameters of the distribution. It is not known how this technique compares to other parametric techniques (eg, maximum likelihood) when the underlying distribution is known. However, this procedure is reasonable when the underlying distribution is not known. The scheme has been tested against known distributions with excellent results.