Convergence properties of direct search methods for stochastic optimization

Simulation is widely used to evaluate the performance and optimize the design of a complex system. In the past few decades, a great deal of research has been devoted to solving simulation optimization problems, perhaps owing to their generality. However, although there are many problems of practical interests that can be cast in the framework of simulation optimization, it is often difficult to obtain an understanding of their structure, making them very challenging. Direct search methods are a class of deterministic optimization methods particularly designed for black-box optimization problems. In this paper, we present a class of direct search methods for simulation optimization problems with stochastic noise. The optimization problem is approximated using a sample average approximation scheme. We propose an adaptive sampling scheme to improve the efficiency of direct search methods and prove the consistency of the solutions.