DocumentCode
1026428
Title
The sqlLoader Data-Loading Pipeline
Author
Szalay, Alex ; Thakar, Ani R. ; Gray, Jim
Author_Institution
Johns Hopkins Univ., Baltimore
Volume
10
Issue
1
fYear
2008
Firstpage
38
Lastpage
48
Abstract
Using a database management system (DBMS) is essential to ensure the data integrity and reliability of large, multidimensional data sets. However, loading multiterabyte data into a DBMS is a time-consuming and error-prone task that the authors have tried to automate by developing the sqlLoader pipeline-a distributed workflow system for data loading.
Keywords
SQL; data integrity; pipeline processing; relational databases; data integrity; database management system; distributed workflow system; error-prone task; multidimensional data set; sqlLoader data-loading pipeline; Data mining; Database systems; Image converters; Image databases; Information retrieval; Multidimensional systems; Observatories; Pipelines; Publishing; Warehousing; SDSS; Sloan Digital Sky Survey Science Archive; astronomy; database management systems; large-scale databases;
fLanguage
English
Journal_Title
Computing in Science & Engineering
Publisher
ieee
ISSN
1521-9615
Type
jour
DOI
10.1109/MCSE.2008.18
Filename
4418768
Link To Document