Title :
A parallel Hyper-Surface Classifier for high dimensional data
Author :
He, Qing ; Wang, Qun ; Du, Chang-Ying ; Ma, Xu-dong ; Shi, Zhong-zhi
Author_Institution :
Key Lab. of Intell. Inf. Process., CAS, Beijing, China
Abstract :
The enlarging volumes of data resources produced in real world makes classification of very large scale data a challenging task. Therefore, parallel process of very large high dimensional data is very important. Hyper-Surface Classification (HSC) is approved to be an effective and efficient classification algorithm to handle two and three dimensional data. Though HSC can be extended to deal with high dimensional data with dimension reduction or ensemble techniques, it is not trivial to tackle high dimensional data directly. Inspired by the decision tree idea, an improvement of HSC is proposed to deal with high dimensional data directly in this work. Furthermore, we parallelize the improved HSC algorithm (PHSC) to handle large scale high dimensional data based on MapReduce framework, which is a current and powerful parallel programming technique used in many fields. Experimental results show that the parallel improved HSC algorithm not only can directly deal with high dimensional data, but also can handle large scale data set. Furthermore, the evaluation criterions of scaleup, speedup and sizeup validate its efficiency.
Keywords :
decision trees; learning (artificial intelligence); parallel programming; pattern classification; MapReduce framework; classification algorithm; data resources; decision tree idea; dimension reduction; ensemble techniques; high dimensional data; parallel hyper-surface classifier; parallel programming technique; HSC; Machine learning; MapReduce; PHSC; Parallel classification;
Conference_Titel :
Knowledge Acquisition and Modeling (KAM), 2010 3rd International Symposium on
Conference_Location :
Wuhan
Print_ISBN :
978-1-4244-8004-3
DOI :
10.1109/KAM.2010.5646172