ProAPT: Projection of APTs with Deep Reinforcement Learning

The highest level in Endsley’s situation awareness model is called projection, when the status of elements in the environment is predicted shortly. In cybersecurity situation awareness, the projection for an Advanced Persistent Threat (APT) requires predicting the following step of the APT. So far, reinforcement learning has not been used to project the following step of APTs. In reinforcement learning, the agent uses the previous states and actions to approximate the best action of the current state. When the number of states and actions is abundant, the agent employs a neural network to approximate the best action of each state. This paper presents a deep reinforcement learning system to project the following step of APTs. As there is some relation between attack steps, we employ the Long Short Term Memory (LSTM) method to approximate the best action of each state. In our proposed system, we project the following steps of APTs based on the current situation. We have evaluated our proposed deep reinforcement learning system on DAPT2020 and SCVIC-APT-2021 datasets. Based on the evaluations performed on the mentioned datasets, four criteria F1, precision, recall, and loss were obtained: 0.9533, 0.9352, 0.97, 0.0143 for the DAPT2020 dataset and 0.9585, 0.9397, 0.978, 0.0124 for SCVIC-APT-2021 dataset, respectively. Although there is no previous research on using reinforcement learning for APT projection, our results compared to the previous supervised and unsupervised methods proposed for multi-step attack projections indicate appropriate functioning.