A neural network approach for a single-camera based Artificial Compound Eye System (ACES)

This work studies an Artificial Compound Eye System (ACES) which consists of a camera, a main lens, and a lenslet array. Similar to the compound eye systems in arthropods, one single shot of the ACES can simultaneously capture the visual image of an object from hundreds of different viewpoints, which essentially record the object´s 3D information. Due to the special optics of the ACES, traditional imaging models based on a pinhole assumption can not be directly applied to the system. In this work, we propose to use artificial neural network (NN) to model the mapping from the image observation of the ACES to the object´s 3D information. To deal with the incomplete observation problem in the ACES, a sliding frame based method is proposed to construct the input and target data to train the neural network. Experimental results show that with the proposed method, position of an object in 3D can be effectively extracted from a single image input and the object´s motion in 3D can be extracted from a 2D optical flow. This work may inspire related efforts in exploring the ACES for novel 3D imaging applications.