Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task

We describe a hybrid brain computer interface that integrates gaze information from an eye tracker with brain activity information measured by electroencephalography (EEG). Users explicitly control the end effector of a robot arm to move in one of four directions using motor imagery to perform a pick and place task. Measurements of the natural eye gaze behavior of subjects is used to infer the instantaneous intent of the users based on the past gaze trajectory. This information is integrated with the output of the EEG classifier and contextual information about the environment probabilistically using Bayesian inference. Our experiments demonstrate that subjects can achieve 100% task completion within three minutes and that the integration of EEG and gaze information significantly improves performance over either cue in isolation.