Density estimation by the penalized combinatorial method

Let f be an unknown multivariate density belonging to a prespecified parametric class of densities, Fk, where k is unknown, but Fk⊂Fk+1 for all k and each Fk has finite Vapnik–Chervonenkis dimension. Given an i.i.d. sample of size n drawn from f, we show that it is possible to select automatically, and without extra restrictions on f, an estimate fn,k̂ with the property that E{∫| fn,k̂−f |}=O(1/n). Our method is inspired by the combinatorial tools developed in Devroye and Lugosi (Combinatorial Methods in Density Estimation, Springer, New York, 2001) and it includes a wide range of density models, such as mixture models or exponential families.