The hidden structure of image datasets

We propose the pdf of W, where W is the normalized withinss after a 1D random projection, as a way to visualize the amount of structure contained in a set of images. Using this pdf, we show that real image datasets tend to have a lot of structure and that part of that structure is highly likely to be captured by a 1D random projection. According to our experiments, the structure of image datasets does not appear to be compatible with that of clusters. Nevertheless, the high degree of structure in image sets leads to an efficient and effective way of clustering image datasets using 1D random projections.