A Fully Distributed Collection Technology for Mass Simulation Data

Massive data have been generated in large-scale distributed simulation based on HLA. The data collection in passively subscribing way cannot meet the requirements of data integrity and simulation scalability. Based on the analysis of existing problems, a new fully distributed collection method was proposed. Two collection architectures were presented corresponding to the post event and near real time data analysis. By adopting the big data processing framework called Hadoop, most of the collected data were processed at local and need not to gather the whole data together from distributed nodes. As a result the network traffic was saved at simulation runtime and the simulation resources could be reused for post analysis. Furthermore, the multileveled memory buffer and resource scheduling based on fuzzy reasoning were used to reduce the impact on performance of federate application. Finally, the testing results on prototype demonstrated that the proposed approach is effective and efficient.