An Open Job Scheduling Service for Large-Scale Data Processing

Author

Xue, Zhenghua ; Li, Jianhui ; Zhou, Yuanchun ; Zhang, Yang ; Shen, Geng

Author_Institution

Comput. Network Inf. Center, Beijing, China

fYear

2011

fDate

12-13 Oct. 2011

Firstpage

27

Lastpage

31

Abstract

Scientific Data Grid (SDG) of Chinese Academy of Sciences aims at integrating distributed scientific data, providing transparent data access mechanism and efficient data analysis, processing and visualization services. SDG Job Scheduler (SDGJS) adopts an open and service-oriented framework. The scheduling policy of SDGJS considers both performance of computing nodes and distribution of large data so that it can achieve an efficient job processing. SDGJS has successfully served for some data grid applications.

Keywords

data analysis; data visualisation; grid computing; scheduling; Chinese Academy of Sciences; SDG job scheduler; data access mechanism; data analysis; data visualization services; distributed scientific data; large-scale data processing; open job scheduling service; scientific data grid; service-oriented framework; Authentication; Computer architecture; Distributed databases; Middleware; Monitoring; Processor scheduling; Statistical analysis; Grid; Job Schedule; Large-Scale Data;

fLanguage

English

Publisher

ieee

Conference_Titel

Open Cirrus Summit (OCS), 2011 Sixth

Conference_Location

Atlanta, GA

Print_ISBN

978-1-4673-0727-7

Type

conf

DOI

10.1109/OCS.2011.11

Filename

6200553