Analysis and classification of optimisation benchmark functions and benchmark suites

New and existing optimisation algorithms are often compared by evaluating their performance on a benchmark suite. This set of functions aims to evaluate the algorithm across a range of problems and serves as a baseline measurement of how the algorithm may perform on real-world problems. It is important that the functions serve as a good representative of commonly occurring problems. In order to select functions that will make up the benchmark suite, the characteristics and relationships among the functions must be known. This paper characterises the landscapes of two commonly used benchmark suites, and uses these landscape characteristics to obtain a high level view of the current state of benchmark functions. This is done by using a self-organising feature map to cluster and analyse functions based on landscape characteristics. It is found that while there are numerous functions that cover a wide range of characteristics, there are characteristics that are under represented, or not even covered at all. Furthermore, it is discovered that common benchmark suites are composed of functions which are highly similar according to the measured characteristics.