Subset selection of best simulated systems

In this paper, we consider the problem of selecting a subset of k systems that is contained in the set of the best s simulated systems when the number of alternative systems is huge. We propose a sequential method that uses the ordinal optimization to select a subset G randomly from the search space that contains the best simulated systems with high probability. To guarantee that this subset contains the best systems it needs to be relatively large. Then methods of ranking and selections will be applied to select a subset of k best systems of the subset G with high probability. The remaining systems of G will be replaced by newly selected alternatives from the search space. This procedure is repeated until the probability of correct selection (a subset of the best k simulated systems is selected) becomes very high. The optimal computing budget allocation is also used to allocate the available computing budget in a way that maximizes the probability of correct selection. Numerical experiments for comparing these algorithms are presented.