DocumentCode
2182313
Title
Partitioning real-time ETL workflows
Author
Simitsis, Alkis ; Gupta, Chetan ; Wang, Song ; Dayal, Umeshwar
Author_Institution
HP Labs., Palo Alto, CA, USA
fYear
2010
fDate
1-6 March 2010
Firstpage
159
Lastpage
162
Abstract
Many organizations are aiming to move away from traditional batch processing ETL to real-time ETL (RT-ETL). This move is motivated by a need to analyze and take decisions on as fresh a data as possible. The RT-ETL engines operate on the abstraction of data flow executed on parallel architectures. For high throughput and low response times, there is a need for partitioning the data over the large number of nodes in the engine. In this paper, we consider the problem of partitioning realtime ETL flows and we propose a high level architecture for that.
Keywords
batch processing (computers); data flow computing; workflow management software; batch processing; data flow execution; high level architecture; parallel architectures; real-time ETL workflows; Costs; Data warehouses; Decision making; Delay; Design optimization; Fault tolerance; Humans; Maintenance; Merging; Real time systems;
fLanguage
English
Publisher
ieee
Conference_Titel
Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on
Conference_Location
Long Beach, CA
Print_ISBN
978-1-4244-6522-4
Electronic_ISBN
978-1-4244-6521-7
Type
conf
DOI
10.1109/ICDEW.2010.5452754
Filename
5452754
Link To Document