An approach to conceptual modelling of ETL processes

In this paper, we propose a new conceptual model based on the visualization of data flow showing transformations of records accompanied with attribute transformations. A data warehouse development relies on the development of Extract, Transform, and Load (ETL) processes responsible for extracting data from transactional systems, data quality checks, transformation and conversion and loading data to target tables or cubes in a data warehouse. During the planning and design phases for data warehouse, the ETL conceptual model should be developed not only to show an overview of the whole process, but also to map sources, targets and necessary transformations, verify transformation logic, ensure that requirements are met and the needed structures and content exist. Once created, the conceptual model can be used for development of an ETL process, maintenance/optimization and redesign of a process due to new business requirements and database schema changes, all of which makes it a valuable addendum to data warehouse documentation.