Title :
An Adaptive Markov Game Model for Threat Intent Inference
Author :
Shen, Dan ; Chen, Genshe ; Cruz, Jose B., Jr. ; Kwan, Chiman ; Kruger, Martin
Author_Institution :
Intelligent Autom., Inc., Rockville
Abstract :
In an adversarial military environment, it is important to efficiently and promptly predict the enemy´s tactical intent from lower level spatial and temporal information. In this paper, we propose a decentralized Markov game (MG) theoretic approach to estimate the belief of each possible enemy course of action (ECOA), which is utilized to model the adversary intents. It has the following advantages: (1) It is decentralized. Each cluster or team makes decisions mostly based on local information. We put more autonomies in each group allowing for more flexibilities; (2) A Markov decision process (MDP) can effectively model the uncertainties in the noisy military environment; (3) It is a game model with three players: red force (enemies), blue force (friendly forces), and white force (neutral objects); (4) Correlated-Q reinforcement learning is integrated. With the consideration that actual value functions are not normally known and they must be estimated, we integrate correlated-Q learning concept in our game approach to dynamically adjust the payoffs function of each player. A simulation software package has been developed to demonstrate the performance of our proposed algorithms. Simulations have verified that our proposed algorithms are scalable, stable, and satisfactory in performance.
Keywords :
Markov processes; game theory; inference mechanisms; learning (artificial intelligence); military computing; Markov decision process; adaptive Markov Game model; adversarial military environment; correlated-Q reinforcement learning; enemy Course of Action; military environment; payoffs function; simulation software package; threat intent inference; Automation; Decision making; Estimation theory; Game theory; Learning; Predictive models; Software algorithms; Software packages; Uncertainty; Working environment noise;
Conference_Titel :
Aerospace Conference, 2007 IEEE
Conference_Location :
Big Sky, MT
Print_ISBN :
1-4244-0524-6
Electronic_ISBN :
1095-323X
DOI :
10.1109/AERO.2007.352800