A distributed inverse distance weighted interpolation algorithm based on the cloud computing platform of Hadoop and its implementation

A centralized inverse distance weighted interpolation (IDW) method is simple and widely used, but it is difficult to meet the requirements of mass data processing. The cloud computing technology of Hadoop has the advantages of simple application portability, high system reliability and node dynamic load balancing. The extension of the centralized IDW to the distributed version based on Hadoop is one of the effective ways to deal with massive data processing requirements. This paper presented a distributed algorithm IDW under the MapReduce framework of the Hadoop technology. The core ideas of the algorithm are: (1) the data set to be interpolated is divided into multiple sub-data sets, and each of Map tasks run the serial IDW interpolation algorithm to interpolation a subset of the data set; (2) the Reduce task merges the interpolation results by all map tasks, and outputs the final result. Experimental results shown that the distributed IDW algorithm had good acceleration performance for large-scale data sets, and significantly improve the computational efficiency of spatial interpolation.