TIFA: Enabling Real-Time Querying and Storage of Massive Stream Data

With the proliferation of network applications and user groups, the network stream data scale are getting larger and larger. To manage and analyze the massive data to get better service for end users (a.k.a. network optimization) or understanding of impacts of new network applications are attracting many researchers. This paper focuses on the storage and real-time retrieval of network traffic for analysis. We address this problem by integrating Time Machine and FastBit as a real-time querying and storage of massive stream data system, named TIFA. TIFA is designed for operation in Gbps network, accompanied with UTM as a module or an independent device. It leverages multiple large volume storages, and can achieve Ten TB level in volume. It has been proved useful in the performance of querying and storage of massive stream data in the experiments.