Random subspaces of the instance and principal component spaces for ensembles

In machine learning accurate predictors may be obtained by combining predictions of an ensemble of accurate and diverse predictors. Ensembles are efficiently constructed with the random subspace method (RSM) performed in the instance or in the principal components (PCs) spaces. In this paper, we extend RSM to explore the synergy in the characteristics of these two spaces, with a method referred to as RSM-IPCS. Using 24 datasets from the UCI machine learning repository, we show an enhanced performance of RSM-IPCS in comparison to the original RSM and RSM in PCs space, in terms of higher accuracy and smaller variances. Since RSM-IPCS exhibited at least a similar performance to the best method in a separate space, it opens the way for optimization of ensembles based on the combination of multiple spaces.