Local dagging of decision stumps for regression and classification problems

Numerous data mining problems involve an investigation of relationships between features in heterogeneous datasets, where different prediction models can be more appropriate for different regions. We propose a technique of dagging localized weak learners. We recognize local regions having similar characteristics and then build local experts on each of these regions describing the relationship between the data characteristics and the target value. We performed a comparison with other well known combining methods on standard classification and regression benchmark datasets using decision stump as based learner, and the proposed technique produced the most accurate results.