Title :
Different File Systems Data Access Support on MapReduce
Author :
Zeng Dadan ; Chen Zhebing ; Wang Jianpu ; Zhou Minqi ; Zhou Aoying
Author_Institution :
Massive Comput. Inst., East China Normal Univ., Shanghai, China
Abstract :
In this paper, the interface between MapReduce and different storage systems is proposed. MapReduce based computing platform can access various file systems by the interface without modifying the existing computing system so that to simplify the construction of distributed applications. Different file systems can be configured and get quick switch by the interface. The interface is also integrated in Hadoop to make the experiments. The results show the interface can achieve different storage systems switched at the same time the data access efficiency gets better with increasing data volume. Further experiment with larger data volume will be done and the deeper development of the interface is being taken in the computing platform of our work.
Keywords :
distributed databases; information retrieval; MapReduce system; data access support; distributed computing; file systems access; Clouds; Computer interfaces; Distributed computing; Distributed control; File systems; Internet; Parallel processing; Pipelines; Switches; Writing;
Conference_Titel :
Computational Intelligence and Software Engineering, 2009. CiSE 2009. International Conference on
Conference_Location :
Wuhan
Print_ISBN :
978-1-4244-4507-3
Electronic_ISBN :
978-1-4244-4507-3
DOI :
10.1109/CISE.2009.5363360
