Orthogonal transform of SIFT descriptors and the effect on size and performance of Fisher Vectors in visual search

Fisher Vectors have shown great capability for visual search. Their main drawback is their high dimensionality. We propose several methods to reduce the size of the Fisher Vectors by applying different preprocessing steps and dimension reduction techniques to SIFT descriptors. Also, we investigate the effects of PCA and DCT transforms employed on SIFT descriptors and the resulting improvement for an image retrieval application. We show that DCT transformed SIFT descriptors show superiority over other methods and that dimension reduction to 32 dimensions is possible as well as a reduction of number of clusters used for Fisher Vector generation without too much loss in the mean average precision.