Integrating and optimizing transactional memory in a data mining middleware

As the size of available datasets in various domains is growing rapidly, there is an increasing need for scaling data mining implementations. Coupled with the current trends in computer architecture, where scaling only seems possible with effective utilization of the increasing number of cores, this is leading to a programmability and performance challenge for data mining applications on emerging multi-core architectures. Recently, Software Transactional memory (STM) has been gaining popularity as a viable tool for easing programmability on shared memory machines. This paper focuses on utilizing, optimizing, and evaluating STM for data mining applications on multi-core architectures. The specific contributions of this paper are three-fold: (1) An existing STM algorithm (Transactional Locking II) has been integrated with a parallel data mining middleware, FREERIDE. This enables transparent use of the STM technique by any application developed using this middleware. (2) We have developed a new Hybrid Replication- Transactional Memory scheme, which substantially reduces the memory overhead of a replication scheme, while also reducing the number of conflicts and aborts in the STM technique, and (3) We have performed a comprehensive performance evaluation of STM techniques, where they have been compared with a replication-based scheme (which may not be scalable with increasing number of cores), and a highly optimized locking scheme. Our results show that, both STM and HyRepSTM techniques are competitive with other schemes in most cases. Also, the Hybrid Replication-Transactional memory scheme substantially reduces the number of aborts and conflicts when the number of concurrent threads are high.