Statistical analysis tools for regression testing of SimSET output

SimSET (a Simulation System for Emission Tomography) is widely used for studying PET and SPECT. Techniques to increase simulation efficiency such as importance sampling (IS) have recently been refined with the goal of increasing the efficiency of SimSET by a factor of 20. In order to ensure that IS and other efficiency optimizations do not introduce artifacts in the data, we have developed a new set of tools aimed at confirming equivalence of output from the simulation tool for each revision of the software. Since the efficiency improvement is achieved by performing fewer computations for each detected event, it is not possible to reproduce datasets exactly (by starting with the same random seed) and therefore statistical techniques must be used. In this paper we compare the conventional t-test that is the current standard test, with the chi-squared goodness of fit test and a visual test of the t-statistic.