Hotspot Prediction and cache in distributed stream-processing storage systems

Storage performance is critical in today´s distributed stream-processing systems. One approach to improve the performance is to use hotspot attribute in object-based storage systems. This paper discusses hotspot classification and identification, and then presents an object hotspot prediction model (OHPM) to dynamically predict hotspots. Based on this model, we discuss an efficient hotspot caching strategy to improve the performance. To demonstrate the effectiveness of our proposed approach, we have developed a prototype of hotspot attribute-managed storage system (HASS) by extending object-based storage device (OSD) file system and iSCSI protocols. Experimental results show that the HASS improves the throughput by up to 62% and reduces the disk I/O by as much as 25% in our VoD tests by integrating our object hotspot prediction and cache approaches.