Hash in place with memory shifting: datacube computation revisited

A datacube on n attributes requires the computation of an aggregation function over all groups generated by 2ⁿ interelated GROUP-BYs. Even n is not very large, and the computation could be very expensive if the database involved is large. Although a number of algorithms with various optimization techniques have been proposed, accurate estimation of memory requirement and efficient use of the available memory remain difficult issues. The difficulty of estimating memory requirement comes from data skews. We present a novel hash based approach for datacube computation. The approach effectively uses the available memory to maintain a minimum number of hash tables required for computing related cuboids and manages memory dynamically by shifting memory pages among hash tables. Therefore, no priori memory requirement estimation is necessary and all memory available can be fully utilized