Assessing Robustness of Optimisation Performance for Problems With Expensive Evaluation Functions

In complex engineering problems, the objective functions can be very slow to evaluate, restricting the optimisation process to only a few hundred objective calculations. Often the optimisation process can only be performed once, requiring a good solution from the single run. Thus we need a robust approach to algorithm development and tuning. This paper introduces a new metric for quantifying the performance of different algorithms on different test functions relative to the range of performance expected from a random search. As a random search is a repeatable benchmark for any objective function, the metric can be applied as an absolute, rather than relative metric. The metric allows the best, worst and median performance of different algorithms to be compared directly, even for optimisation runs with only tens of evaluations. Additionally a new optimisation algorithm, based on a Voronoi decomposition of the decision space, is presented that provides reliable optimisation performance, but with a very limited number of function evaluations. The paper evaluates the performance of the new algorithm with the new metric on a range of surfaces and against a typical evolutionary approach.