DocumentCode :
3717362
Title :
SciSpark: Applying in-memory distributed computing to weather event detection and tracking
Author :
Rahul Palamuttam;Renato Marroqu?n Mogrovejo;Chris Mattmann;Brian Wilson;Kim Whitehall;Rishi Verma;Lewis McGibbney;Paul Ramirez
Author_Institution :
NASA Jet Propulsion Laboratory California Institute of Technology Pasadena, CA, USA
fYear :
2015
Firstpage :
2020
Lastpage :
2026
Abstract :
In this paper we present SciSpark, a Big Data framework that extends Apache™ Spark for scaling scientific computations. The paper details the initial architecture and design of SciSpark. We demonstrate how SciSpark achieves parallel ingesting and partitioning of earth science satellite and model datasets. We also illustrate the usability and extensibility of SciSpark by implementing aspects of the Grab ´em Tag ´em Graph ´em (GTG) algorithm using SciSpark and its Map Reduce capabilities. GTG is a topical automated method for identifying and tracking Mesoscale Convective Complexes in satellite infrared datasets.
Keywords :
"Arrays","Sparks","Libraries","Clouds","Meteorology","File systems"
Publisher :
ieee
Conference_Titel :
Big Data (Big Data), 2015 IEEE International Conference on
Type :
conf
DOI :
10.1109/BigData.2015.7363983
Filename :
7363983
Link To Document :
بازگشت