Cogset vs. Hadoop: Measurements and Analysis

Cogset is an efficient and generic engine for reliable storage and parallel processing of data. It supports a number of high-level programming interfaces, including a MapReduce interface compatible with Hadoop. In this paper, we evaluate Cogset´s performance as a MapReduce engine, comparing it to Hadoop. Our results show that Cog set generally outperforms Hadoop by a significant margin. We investigate the causes of this gap in performance and demonstrate some relatively minor modifications that markedly improveHadoop´s performance, closing some of the gap.