Multi-scale feature based salient environmental sound recognition for machine awareness

Auditory perception of surrounding environment is important to machine awareness. To provide artificial awareness ability for machines, a bio-inspired salient environmental sound detection and recognition method is proposed. The salient sounds are detected by using the auditory saliency map which based on heterogeneous saliency features from visual and acoustic domain. Spectral and temporal saliency features from both power spectral density (PSD) and mel-frequency cepstral coefficients (MFCC) as well as the visual saliency from log-scale spectrogram are applied to yield the final auditory saliency for salient sound detection. To improve the detection accuracy, short-term Shannon entropy (SSE) and a computational inhibition of return (IOR) model are initially proposed to verify the temporal saliency characteristic. The detected salient sounds are classified by using the features which based on the fuzzy vector of spectral energy distribution and MFCC. A two-level classification is presented based on the support vector machine (SVM) for recognition task. Experiments are carried out on the real environmental sound examples. The results show that, over 83% recognition accuracy can be achieved by using proposed fuzzy vector based features, and the overall accuracy of 94.65% can be obtained when combined with MFCC based features.