A Research Overview and Evaluation of Performance Metrics for Self-Organization Algorithms

Self-organization (SO) algorithms are supposed to restructure and reconfigure the system at run-time in order to empower it to fulfill its requirements under uncertain environmental conditions. For this purpose, information about the state of the environment and the system is used in feedback loops to establish a flexible, powerful system. Consequently, the performance of the SO algorithms has a significant effect on the overall performance of the system. Indeed, it is hard to design high-performing SO algorithms, because the environmental conditions the system has to operate in are partially unpredictable at design time. A crucial aid for the development of SO algorithms are tools that enable the evaluation of the algorithms´ performance at design time. These tools could also be used to select the best-fitting algorithm and parametrization for a specific application, among others. We show how existing performance metrics can be applied to SO algorithms by evaluating different partition-based algorithms. Based on these results, we discuss the advantages and limitations of the existing metrics and deduce requirements for performance metrics for SO algorithms.