Probability Collectives in Dynamic Environments: A Study of Controlling the Balance between Exploration and Exploitation of the Search

Probability Collective (PC) is an extension of conventional game theory for distributed optimization by sampling an explicitly parameterized probability distributions over the space of solutions. This parameterization introduces more effective computational models to solve complex systems-level optimization problems. In this paper we present a study of using this collective learning model for adaptive optimization in the context of dynamic environments. Two scenarios of PC in dynamic optimization tasks are investigated: PC1 (original PC settings), PC2 (temperature T - a factor controlling the balance between exploration and exploitation of the search process - is reset to the initial state when an environmental change takes place). By allowing PC to re-explore the search space, we show that PC2 is more adaptive to environmental changes, thereby outperforming the original PC in rate of descent as well as long term extrema-tracking optimization. The study of the PC in changing environments therefore sheds light into how this collective learning methodology advances the current state of research in agent-based models for dynamic optimization problems.