A study of collaborative distributed multi-goal & multi-agent-based systems for large Critical Key Infrastructures and Resources (CKIR) dynamic monitoring and surveillance

In many homeland security and counterterrorism applications, there is a need to secure and protect large Critical Key Infrastructures and Resources (CKIR), such as transportation, aviation, highways, and maritime systems. These usually span hundreds of thousands to millions of miles of roadways, maritime, and/or airways. To achieve the monitoring of such large CKIR systems there is a need to develop cooperative intelligent geographically and computationally distributed multi-agent-based monitoring systems. In such dynamic large environments, where agents could be moving randomly or less randomly to reach their respective goals, agent path planning for autonomous agents becomes challenging. It is desired to have the agents not only be equipped with intelligence to move autonomously and avoid obstacles and each other, but also to be able to learn autonomously how to reach the shortest path to their goal(s) in a minimum amount of time and exploratory trials. In this paper we address issues related to multi-agent reinforcement learning in a distributed-computing-memory environment where the agents have limited and not complete knowledge of their environment(s). We present an architecture for the distributed dynamic agent communication based on the Message Passing Interface (MPI). In particular, the focus and contribution of this paper are two-fold: the analysis of the effects, on the multi-agent system learning and total execution time, of (a) the agent environment size, (b) the agents shared-over-a-network learned information with/from other agents with one same goal versus agents not sharing information among themselves and (c) the agents sharing information when having one same agent goal versus the agents sharing information when having different agent goals. The desired aim is the analysis of the factors and conditions that increase the overall distributed multi-agent system computational performance and learning speed.