Different File Systems Data Access Support on MapReduce

Author

Zeng Dadan ; Chen Zhebing ; Wang Jianpu ; Zhou Minqi ; Zhou Aoying

Author_Institution

Massive Comput. Inst., East China Normal Univ., Shanghai, China

fYear

2009

fDate

11-13 Dec. 2009

Firstpage

1

Lastpage

4

Abstract

In this paper, the interface between MapReduce and different storage systems is proposed. MapReduce based computing platform can access various file systems by the interface without modifying the existing computing system so that to simplify the construction of distributed applications. Different file systems can be configured and get quick switch by the interface. The interface is also integrated in Hadoop to make the experiments. The results show the interface can achieve different storage systems switched at the same time the data access efficiency gets better with increasing data volume. Further experiment with larger data volume will be done and the deeper development of the interface is being taken in the computing platform of our work.

Keywords

distributed databases; information retrieval; MapReduce system; data access support; distributed computing; file systems access; Clouds; Computer interfaces; Distributed computing; Distributed control; File systems; Internet; Parallel processing; Pipelines; Switches; Writing;

fLanguage

English

Publisher

ieee

Conference_Titel

Computational Intelligence and Software Engineering, 2009. CiSE 2009. International Conference on

Conference_Location

Wuhan

Print_ISBN

978-1-4244-4507-3

Electronic_ISBN

978-1-4244-4507-3

Type

conf

DOI

10.1109/CISE.2009.5363360

Filename

5363360