Cloud Resource Management for Image and Video Analysis of Big Data from Network Cameras

With the existence of millions of public network cameras capturing countless events around the world, there is a need for a system to retrieve, save, and analyze the tremendous amount of visual data from the cameras. The knowledge from the data will ultimately help better understand the world. Such a system needs to allocate and manage significant amounts of resources in order to meet the analysis requirements. In order to reduce the overall analysis cost, this paper presents a cloud resource manager that allocates cost-effective cloud instances, monitors and automatically scales the cloud resources. The paper presents a system that uses the proposed resource manager for image and video analysis of the big data from global network cameras. Our experiments show that the resource manager can lead to 13% reduction in cost. The experiments use four analysis programs which represent different workloads in terms of CPU and memory. The experiments show that different cloud instances are more cost-effective for different analysis programs. One experiment analyzes data streams from 1026 cameras simultaneously for six hours using different analysis programs at different frame rates. The experiment analyzes 5.5 million images, totalling 260GB data.