Binocular visual feedback with CNN sensors

This paper describes the use of a CNN sensor to provide visual feedback signals in an active binocular vision system. The sensor outputs are used to control the vergence angles of the two cameras and the tilt angle of the head so that the center pixels of the sensors image the same point in the environment. The sensors contain 2D arrays of phototransistors for image input and CNN-based processing circuits that convolve the input image with filters similar to even and odd Gabor filters. We use the odd and even filter outputs in a phase based algorithm, which enables target motions to be detected with sub-pixel resolution. Experimental results demonstrating the ability of the system to reconstruct target motions in 3D are described