DocumentCode :
2984505
Title :
Ensemble Pruning via Constrained Eigen-Optimization
Author :
Linli Xu ; Bo Li ; Enhong Chen
Author_Institution :
Sch. of Comput. Sci. & Technol., Univ. of Sci. & Technol. of China, Hefei, China
fYear :
2012
fDate :
10-13 Dec. 2012
Firstpage :
715
Lastpage :
724
Abstract :
An ensemble is composed of a set of base learners that make predictions jointly. The generalization performance of an ensemble has been justified both theoretically and in practice. However, existing ensemble learning methods sometimes produce unnecessarily large ensembles, with an expense of extra computational costs and memory consumption. The purpose of ensemble pruning is to select a subset of base learners with comparable or better prediction performance. In this paper, we formulate the ensemble pruning problem into a combinatorial optimization problem with the goal to maximize the accuracy and diversity at the same time. Solving this problem exactly is computationally hard. Fortunately, we can relax and reformulate it as a constrained eigenvector problem, which can be solved with an efficient algorithm that is guaranteed to converge globally. Convincing experimental results demonstrate that this optimization based ensemble pruning algorithm outperforms the state-of-the-art heuristics in the literature.
Keywords :
combinatorial mathematics; eigenvalues and eigenfunctions; learning (artificial intelligence); optimisation; combinatorial optimization problem; constrained eigen-optimization; constrained eigenvector problem; ensemble learning method; ensemble pruning; Accuracy; Bagging; Complexity theory; Optimization; Predictive models; Training; Vectors; ensemble pruning; optimization;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Data Mining (ICDM), 2012 IEEE 12th International Conference on
Conference_Location :
Brussels
ISSN :
1550-4786
Print_ISBN :
978-1-4673-4649-8
Type :
conf
DOI :
10.1109/ICDM.2012.97
Filename :
6413857
Link To Document :
بازگشت