A Parallel Algorithm for Datacleansing in Incomplete Information Systems Using MapReduce

Data cleansing is an important process of data mining. It is the key technology for ensuring the quality of the data. Classical data pre-processing technique has limitation in processing massive data with missing information, and sometimes it can not obtain precise and reasonable results, which leads to low-quality data. To this end, through deep analysis of the classical pre-processing, combining with the MapReduce programming model, A parallel algorithm for data cleansing in incomplete information systems using MapReduce is put forward to process the massive data with missing information. Finally, the new algorithm is applied to incomplete decision information system, and the analysis results show that the new algorithm is effective.