A co-robotic assistant capable of object selection and search via a brain machine interface

Co-robotic assistants, or Cobots can improve the quality of life for individuals with locked-in syndrome (LIS) by allowing augmented control over their surroundings. Implemented in collaboration between the Boston University Neuromorphics Lab and Neural Prosthesis Lab, this work provides a proof of concept of an autonomous robot coupled with a non-invasive brain machine interface (BMI). The system uses Steady State Visually Evoked Potential (SSVEP) for target selection and a massively parallel neural network that models functionality of the primate “where” and “what” visual pathways. The simulated visual processes perform object recognition on images streamed from the Cobot equipped with a pan-and-tilt camera. In this paper, we describe a subcomponent of the system designed to allow the neural network to learn the identity of objects, the user to select a target object and the Cobot to perform autonomous visual investigation.