Adaptively choosing the best procedure for selecting the best system

The paper presents a method that uses initial sample data to choose between statistical procedures for identifying the simulated system with the best (maximum or minimum) expected performance. The method chooses the procedure that minimizes the additional number of simulation replications required to return a pre-specified probability guarantee. This problem may be encountered after a heuristic search procedure has been applied in a simulation-optimization context. In this setting, initial samples from each system may already have been taken, but because of stochastic variation, the system with the best sample mean at the end of the search procedure may not be the true best system encountered during the search. Empirical work in previous papers suggests that the relative number of additional replications required by existing procedures depends on factors such as the configuration of the systems´ means and their variances that may be unknown prior to initial data collection. These results motivated the approach taken in the paper, where we postpone the choice between statistical procedures until after observing the initial data