Algorithm comparison for manufacturing scheduling problems

The performance of heuristic algorithms for combinatorial optimization is often highly sensitive to problem instances. A specialized heuristic algorithm may perform exceptionally well on a particular set of instances while fail to produce acceptable solutions on others. This paper proposes a formal method for comparing and selecting heuristic algorithms in real-time given a desired confidence level and a particular set of problem instances. We formulate this algorithm selection problem as a stochastic optimization problem. Two approaches for optimization, ordinal optimization and computing budget allocation, are applied to solve this algorithm selection problem. Computational testing on a set of statistical clustering algorithms in the IMSL library is conducted, which demonstrates that our method can effectively compare and select algorithms that are expected to perform well on given problem instances