MIPr - a framework for distributed image processing using Hadoop

Nowadays, the sizes of image collections are increasing dramatically and reaching petabytes of data. Such large volumes cannot be analyzed on personal computer within a reasonable time. Therefore, processing of modern image collections requires distributed computing. This paper presents a MapReduce Image Processing framework (MIPr), which provides the ability to use distributed computing for image processing. MIPr is based on MapReduce and its open source implementation Apache Hadoop. MIPr provides various forms of image representations in Hadoop internal format and the input/output tools for integration of image processing into Hadoop data workflow. The image formats in the MIPr framework are based on the popular image processing libraries. Furthermore, the MIPr includes the high-level Image processing API for developers who are not familiar with Hadoop. This API allows to create sequential functions that process one image or a group of related images. The MIPr framework applies such functions to the large amount of images in parallel. In addition, MIPr includes MapReduce implementations of popular image processing algorithms, which can be used for distributed image processing without any software development. The MIPr framework significantly simplifies image processing in Hadoop distributed environment.