An ensemble of classifiers approach for the missing feature problem

A new learning algorithm is introduced that can accommodate data with missing features. The algorithm uses an ensemble of classifiers approach. The classifiers in the ensemble are trained with random subsets of the total number of available features. The approach takes advantage of the basic assumption that an unknown subset of the features is in fact adequate for the classification, or in other words, that are redundant, and possibly irrelevant features in the data. This assumption is in general true for most practical applications. We empirically show that if a certain number of networks produce a particular classification performance using all of the features, then the same classification performance can be reached even if some features are missing, as long as the same number of usable networks can be generated with the missing features. The proposed approach has its roots in the incremental learning algorithm, Learn⁺⁺ which seeks to learn new information that is provided by additional datasets that may later become available, even when such data introduce new classes. We have modified the Learn⁺⁺ algorithm for addressing the missing feature problem. The proposed algorithm showed surprisingly remarkable performance on three real-world applications, with up to 10% of the features missing in the validation / field data.