A tournament framework for the ranking and selection problem

A tournament can be broadly defined as a procedure that ranks agents, where they exhibit their performance in a noisy environment. By observing the agents´ performance, the organizer computes their ranking and rewards them according to the revealed ranking. The organizer´s challenge is therefore to determine the optimal tournament format that identifies the best agent in the most effective fashion. Tournaments thus provide a natural framework for ranking and selection (R&S) via simulation, which represents a set of approaches developed to complement the modeling flexibility of simulation with the efficiency of statistical techniques for effective decision making. In this paper, following the introduction of a general framework to represent various tournament formats and to assess their predictive power, we will report preliminary experimental results on the effectiveness of tournaments in identifying the best simulated system with the desired probability of correct selection in the presence of costs.