HPCA: A node selection and scheduling method for Hadoop MapReduce

Big data is the technology which is designed to handle both structured and unstructured data which has high intensity. Hadoop and MapReduce are two important aspects of big data. Task assignment in MapReduce is done through scheduling algorithms. Scheduling algorithms assign the tasks to a selected data node. Selection of a healthy and available data node to perform the Map and reduce is done based on the availability and the location of the data on which the processing should be done. Creating an algorithm for the node selection is essential to discipline and optimize and improve the performance of the MapReduce. The proposed Health, Priority, Capacity and Availability based Node selection algorithm [HPCA based Node Selection Algorithm] creates a queue of the nodes that are available for accepting the new tasks through scheduling algorithms. This algorithm optimizes the node selection task and provides better performance. It also introduces a failover mechanism to handle the tasks that fail during the execution.