A framework study of ETL processes optimization based on metadata repository

Author

Li, Lunan

Author_Institution

Sch. of Glorious Sun Bus. & Manage., Donghua Univ., Shanghai, China

Volume

6

fYear

2010

fDate

16-18 April 2010

Abstract

ETL plays a significant role in DW construction. Basically, ETL management is rooted in metadata management of the ETL processes. A metadata management system with good design can highly improve the ETL efficiency. However, the large amount and the wide distribution of metadata in ETL processes cause the mismanagement of metadata, which hasn´t got a good solution yet. To deal with this problem, the paper suggests intensively manage ETL by metadata repository. Metadata repository can show DBA the metadata in a direct, simple, and centred way, and makes metadata more easy to understand, therefore metadata management becomes more direct, simple, and centred. Compared with the common one, ETL processes based on metadata repository have a much better optimization effect.

Keywords

data warehouses; meta data; optimisation; DW construction; ETL management; ETL processes optimization; metadata distribution; metadata management; metadata repository; Chaos; Data mining; Data structures; Data warehouses; Databases; Design optimization; Memory; Standards development; Sun; ETL; data warehouse; metadata; metadata repository;

fLanguage

English

Publisher

ieee

Conference_Titel

Computer Engineering and Technology (ICCET), 2010 2nd International Conference on

Conference_Location

Chengdu

Print_ISBN

978-1-4244-6347-3

Type

conf

DOI

10.1109/ICCET.2010.5486338

Filename

5486338