Title :
Efficient data preprocessing for genetic-fuzzy mining with MapReduce
Author :
Tzung-Pei Hong ; Yu-Yang Liu ; Min-Thai Wu ; Chun-Wei Tsai
Author_Institution :
Dept. of Comput. Sci. & Inf. Eng., Nat. Kaohsiung Univ., Kaohsiung, Taiwan
Abstract :
Genetic-fuzzy data mining can successfully find out linguistic association rules and appropriate membership functions close to human concepts from quantitative transactions, and thus becomes a promising research field in these years. It repeatedly uses fuzzy frequent 1-itemsets to evaluate fitness values of chromosomes, which is very time-consuming. In this paper, we propose a MapReduce preprocessing approach to efficiently transform given quantitative transaction data into pairs of items and quantity lists to increase the performance of genetic-fuzzy mining. The MapReduce architecture totally fits the conversion due to its characteristics of key-value format. Experimental results also show the effect of the proposed approach.
Keywords :
data handling; data mining; fuzzy set theory; genetic algorithms; parallel processing; MapReduce architecture; MapReduce preprocessing approach; chromosome fitness value evaluation; data preprocessing; fuzzy frequent 1-itemsets; genetic-fuzzy data mining; human concept; key-value format; linguistic association rules; membership function; quantitative transaction; transaction data; Algorithm design and analysis; Association rules; Computer science; Genetic algorithms; Indexes;
Conference_Titel :
Consumer Electronics - Taiwan (ICCE-TW), 2015 IEEE International Conference on
Conference_Location :
Taipei
DOI :
10.1109/ICCE-TW.2015.7217045
