ATLAS grid workload on NDGF resources: Analysis, modeling, and workload generation

Author

Karpenko, D. ; Vitenberg, Roman ; Read, A.L.

Author_Institution

Dept. of Phys., Univ. of Oslo, Oslo, Norway

fYear

2012

fDate

10-16 Nov. 2012

Firstpage

1

Lastpage

11

Abstract

Evaluating new ideas for job scheduling or data transfer algorithms in large-scale grid systems is known to be notoriously challenging. Existing grid simulators expect to receive a realistic workload as an input. Such input is difficult to provide in absence of an in-depth study of representative grid workloads. In this work, we analyze the ATLAS workload processed on the resources of NDG Facility. ATLAS is one of the biggest grid technology users, with extreme demands for CPU power and bandwidth. The analysis is based on the data sample with ~1.6 million jobs, 1,723 TB of data transfer, and 873 years of processor time. Our additional contributions are (a) scalable workload models that can be used to generate a synthetic workload for a given number of jobs, (b) an open-source workload generator software integrated with existing grid simulators, and (c) suggestions for grid system designers based on the insights of data analysis.

Keywords

data analysis; grid computing; high energy physics instrumentation computing; position sensitive particle detectors; public domain software; sampling methods; ATLAS grid workload; CPU; NDGF resources; Nordic Data Grid Facility; data analysis; data transfer algorithms; grid simulators; grid system designers; job scheduling; large-scale grid systems; open-source workload generator software; scalable workload models; synthetic workload; Analytical models; Correlation; Data models; Data transfer; Generators; Scheduling;

fLanguage

English

Publisher

ieee

Conference_Titel

High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for

Conference_Location

Salt Lake City, UT

ISSN

2167-4329

Print_ISBN

978-1-4673-0805-2

Type

conf

DOI

10.1109/SC.2012.21

Filename

6468494