Improved Behavior Knowledge Space Combination Method with Observational Learning

A new combination model, which incorporated observational learning with Behavior Knowledge Space (BKS) method, was proposed in this paper. Observational learning is a process with three steps: training, observing, and retraining. The proposed model generated simulated data by observational learning to extend the datasets, so it is effective in solving the small sample size problem that BKS suffers. Experimental investigations were performed on five datasets from the UCI repository. Bias-variance decomposition of the error indicates that observational learning algorithm can reduce both bias and variance. It is shown that, observational learning outperforms the individual base learner and majority voting when base learners are not capable enough for the given task, and classification performance can be improved further by repeat the "observing-retraining" process. Experiments also show that the combination model proposed in this work is superior to the basic BKS method and the BKS method with training dataset enlarged by injecting random noise.