CHAC: An Effective Attribute Clustering Algorithm for Large-Scale Data Processing

Nowadays Hadoop has become a leading architecture for large-scale data processing. One of the efficient ways to accelerate data processing is column-oriented storage technique which has been integrated into Hadoop family recently. However, how to design an appropriate attribute clustering algorithm to achieve optimal data processing performance for column-oriented hadoop system is still a big problem. In this paper, we propose a novel algorithm called CHAC to solve this problem. Both cases of overlapping attribute cluster and non-overlapping attribute cluster are considered in CHAC. In addition, an adjustable parameter is also taken into account to prohibit excessive attribute redundancy via limiting space overhead. The experimental results on TPC-H Benchmark demonstrate the efficiency and effectiveness of the proposed algorithm.