A methodology for integrating and exploiting data mining techniques in the design of data warehouses

Data Warehousing and Data Mining are two mature disciplines in their own right. Yet, they have developed largely separate from each other, despite the fact that techniques developed for pattern recognition such as Clustering and Visualization in the Data Mining discipline have much to offer in the design of Data Warehouses. This is somewhat surprising, given that the two disciplines have broadly the same set of objectives, although the techniques that they employ are admittedly quite different from each other. This may be due to the lack of a suitable methodology for integrating methods such as clustering and pattern visualization into data warehousing design. In this research, we propose such a methodology and report on its application to two case studies involving real world data taken from the UCI Machine Learning repository. We demonstrate how data clustering and visualization methods, working in conjunction with each other can be used to gain new insights and build more meaningful dimensions which may not be obvious to human data warehouse designers.