An optimized ETL fault-tolerant algorithm in data warehouses

Extraction-Transformation-Loading (ETL) plays an important role in data warehouse. Typically, performance is considered the main factor in ETL projects. Actually, faulttolerance and many other aspects influence the results of ETL greatly especially when the time period of projects are long and transformation rules cannot be determined from beginning, such as the situation of changing business logic. To satisfy the fault-tolerance and data validation in such kinds of situation, in this paper, we introduce a fault-tolerant algorithm which gives Redo strategy for different process interrupt scenarios. Moreover, we present a compound refresh mode consisting of full and incremental refresh to guarantee data correctness in changing business logic as well as timely data migration.