An approach to classify visual semantic based on visual encoding with the convolutional neural network

Visual encoding models and visual decoding models are all used for revealing the relationship of external visual stimuli and brain activities. On the view of the goal, visual encoding and decoding are opposite, but they are also complementary. With further researching, we could not use a simple decoding model to excavate more complex information. The combination of visual encoding model and decoding model is becoming a trend of visual research. This paper has built a visual encoding model based on a convolutional neural network and has proposed an approach to select features based on the visual encoding model. The results show that our predicted voxels are similar to the selected real voxels, the correlation coefficients of them are pretty high. On the other hand, the classification results are better than those of t-value method and principal feature analysis (PFA) method. Our approach provides a reference to the combination of visual encoding and decoding models.