The sensitivity of multi-objective optimization algorithm performance to objective function evaluation budgets

When solving multi-objective optimization problems, practitioners desire the most accurate solution possible given the available objective function evaluation (OFE) budget. Since OFE budgets vary largely in practice, it is important to gauge the sensitivity of an optimization algorithm to OFE budget constraints. To this end, a method is presented for measuring the sensitivity of an algorithm´s control parameter values (CPVs) to OFE budgets. Using this method, an OFE budget sensitivity study is conducted for the commonly used non-dominated sorting genetic algorithm (NSGA-II) and the strength Pareto evolution algorithm 2 (SPEA2). It is observed that both SPEA2 and NSGAII are sensitive to OFE budget constraints, with different CPV tuples resulting in optimal performance at different OFE budgets. The results do show the existence of CPV tuples which are both relatively insensitive to OFE budgets and which perform well over a large range of OFE budgets. However these CPV tuples are outperformed by CPV tuples which are optimal for a specific OFE budget. Given these results, it is recommended that when using these algorithms, multi-objective optimization practitioners select CPVs that are well suited to the available OFE budget for the application problem or at least select CPVs shown to have a low sensitivity to OFE budgets.