Mass flow logs analysis system based on Hadoop

As the Internet grows rapidly, analyzing network traffic flow logs become more and more difficult. One main reason is that the size of flow logs is much bigger than ever. In this paper, we proposed a new massive flow logs analyzing system. It is a distributed system based on Hadoop, and it can analyze TBs of data at the same time. We compared our new system with a centralized system, the result shows that our new flow logs analyzing system works much faster and can handle a much bigger dataset.