A SoC with FPGA Landmark Acquisition System for Binocular Visual SLAM

The present paper analyzes the use of an embedded system-on-a-chip (SoC) platform, integrating a multicore ARM processor with FPGA fabric in a single chip, as a stereo vision pre-processing module, used to retrieve depth information from the features to compose 3D landmark points. The Harris and Stephens corner detector is applied to an image pair acquired from a stereo camera setup using a hardware co-processor synthesized in the FPGA to speed up feature extraction and relieve this highly parallelizable process from the main embedded processor. The other tasks that compose the stereo vision processing module, such as image correction from camera calibration, finding a unique descriptor for the detected features, determining the correspondence between POIs in the stereo pair and retrieving depth are solved in software running on the main processor. The proposed hardware implementation enabled the corner extraction task to be performed in about half the time taken solely by the embedded ARM processor, speeding up the vision pre-processing task as a whole.