Thump storage: A management and analysis system for structured big data

Author

Xu Tao ; Fu Ge ; Tan Huaiyuan ; Zhang Hong ; Liu Xinran

Author_Institution

Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China

fYear

2013

fDate

20-22 Dec. 2013

Firstpage

2424

Lastpage

2427

Abstract

Structured big data presents a great challenge to the current platform and technology. It is a crossing field of two technological fields, namely the distributed-storage and parallel-processing of massive data, and relation-oriented database. In this paper, we present a management and analysis system for structured big data called ThumpStorage by integrating the bottom distributed structure of Hadoop distributed file system (HDFS) and the partitioning and scheduling technology of the massive parallel processing (MPP) database. This system shows high efficiency, low latency and high scalability. Finally, we test ThumpStorage by single table query, multi table query and concurrent jobs under different cluster node numbers, and compare with the test result of Hive.

Keywords

Big Data; relational databases; HDFS; Hadoop distributed file system; MPP database; ThumpStorage; analysis system; cluster node numbers; concurrent jobs; distributed storage; distributed structure; management; massive data; massive parallel processing; multitable query; relation-oriented database; scheduling technology; single table query; structured big data; Big data; Distributed databases; Engines; Monitoring; Scalability; Servers; Big Data; HDFS; Hive; MPP; MapReduce;

fLanguage

English

Publisher

ieee

Conference_Titel

Mechatronic Sciences, Electric Engineering and Computer (MEC), Proceedings 2013 International Conference on

Conference_Location

Shengyang

Print_ISBN

978-1-4799-2564-3

Type

conf

DOI

10.1109/MEC.2013.6885442

Filename

6885442