Construction of irregular histograms by penalized maximum likelihood: A comparative study

Theoretical advances of the last decade have led to novel methodologies for probability density estimation by irregular histograms and penalized maximum likelihood. Here we consider two of them: the first one is based on the idea of minimizing the excess risk, while the second one employs the concept of the normalized maximum likelihood (NML). Apparently, the previous literature does not contain any comparison of the two approaches. To fill the gap, we provide in this paper theoretical and empirical results for clarifying the relationship between the two methodologies. Additionally, we introduce a new variant of the NML histogram. For the sake of completeness, we consider also a more advanced NML-based method that uses the measurements to approximate the unknown density by a mixture of densities selected from a predefined family.