Reducing structured Big data benchmark cycle time using query performance prediction model

The paradigm of big data demands either extension of existing benchmarks or building new benchmarks to capture the diversity of data and impact of change in data size and/or system size. This has led to increase in cycle time of benchmarking an application which includes multiple workloads executions on different data sizes. This paper addresses the problem of reducing the benchmark cycle time for structured application evaluation on different data sizes. The paper propose an approach of reducing Big data benchmark cycle time using prediction models for estimating SQL query execution time with data growth. The paper also proposes a model which could be used for efficient tuning of benchmark queries before their executions, to speed up the application evaluation process, on different data sizes. The proposed model estimates structured query execution time for large data size by exploiting data value distribution without actually generating high volume data. The model is validated against three lab implementation of real life applications and TPC-H benchmarks.