Boost a Weak Learner to a Strong Learner Using Ensemble System Approach

The goal of classification learning is to develop a model that separates the data into the different classes, with the aim of classifying new examples in the future. A weak learner is one which takes labeled training examples and produces a classifier which can label test examples more accurately than random guessing. When such weak learner is used directly for classification task then it may not give the better prediction accuracy, due to the limitation and simplicity of single classifier system. On the other hand, multiple classifier systems often known as ensemble based systems, have shown to produce favorable results compared to single-classifier systems. Boosting is one of the most important recent developments in ensemble system, which works by sequentially applying a classification algorithm to re-weighted versions of the training data and then taking a weighted majority vote of the sequence of classifiers. Our experiments demonstrate the underlying weak learner´s ability to achieve a fairly low error rate on the testing data, as well as the boosting algorithm´s ability to reduce the error rate of the weak learner. In our experiment we have used decision stump as a weak learner (classifier) and using the boosting approach, the result demonstrates the improvement in the classifier´s accuracy.