Acoustic attack on keyboard using spectrogram and neural network

Acoustic side channel belongs to one of the oldest side channel and currently, the acoustic attacks are focused on computer keyboards, automated teller machine and internal computer components. Different methods are used for a classification of acoustic traces measured. It primary depends on the fact if the attacker processes the measured data in time or frequency domain. These two approaches use mostly neural networks connected to dictionary using hidden Markov models for an improvement of classification results. We decided for a compromise between the time and frequency domains and we process acoustic trace measured in the time-frequency domain by using a spectrogram. We use the spectrogram as an input of a typical two-layer neural network with the back propagation learning algorithm. This approach is based on a simple algorithm and does not use any other tool to improve classification results. We used widely available laptop with an integrated microphone placed in an office to analyze the potential repeatability and feasibility of the proposed method.