Semantic-based sound retrieval by ERP in rapid serial auditory presentation paradigm

“Semantic gap” is the major bottleneck of semantic-based multimedia retrieval technique in the field of information retrieval. Studies have shown that robust semantic-based image retrieval can be achieved by single-trial visual evoked event related potential (ERP) detection. However, the question remains whether auditory evoked ERP can be utilized to achieve semantic-based sound retrieval. In this paper, we investigated this question in the rapid serial auditory presentation (RSAP) paradigm. Eight BCI-naïve participants were instructed to perform target detection in RSAP sequences with the vocalizations of 8 familiar animals as sound stimuli, and we compared ERP components and single-trial ERP classification performance between two conditions, the target was a predefined specific one, and the targets were different but belonged to the same semantic category (i.e., semantic-based sound retrieval). Although the amplitudes of ERP components (e.g., N2 and P3) and classification performance decreased a little due to the difficulty of the semantic-based sound retrieval tasks, the best two participants still achieved the area under the receive operating characteristic curve (AUC) of single-trial ERP detection more than 0.77. It suggested that semantic-based sound retrieval by auditory evoked ERP was potentially feasible.