Calibrating and Comparing Simulators for Wireless Sensor Networks

In this paper, we present our findings from the calibration and comparison of selected simulators for wireless sensor networks. This work is motivated by our current research on a framework that optimizes a network-based intrusion detection system for a given application. For this purpose, we need a simulator that supports realistic models for topology, antenna, radio propagation, noise, radio, medium access control and energy consumption - factors that can influence the performance of an intrusion detection system, which is intended to be run on the medium access control layer. In the paper, we consider four open-source simulators - Castalia, MiXiM, TOSSIM and WSNet. We compare these simulators and run a set of experiments on MICAz sensor nodes in the indoor and outdoor environment. Based on the data gathered from the real experiments, we calibrate the radio propagation and noise models of Castalia, MiXiM, TOSSIM and WSNet. Also, we calibrate the energy consumption model of Castalia, MiXiM and WSNet according to the MICAz datasheet. We present the results from the simulations and compare them between each other. Even though the simulators are set in the same way, their results significantly differ from each other. In the paper, we discuss possible reasons of the differences.