Image tracking of laparoscopic instrument using spiking neural networks

Minimally Invasive Surgery (MIS) has become more and more popular in recent years. An endoscopic image tracking system will assist surgeons to adjust the field of view autonomously in MIS. In this paper, we propose a novel image tracking algorithm based on natural features of surgical instruments. We suggest to use texture and geometric features in laparoscopic instrument imagery and to adopt a spiking neural network approach for object detection; considering color will be affected by lighting and the white balance conditions in the endoscope imagery. To enhance tracking performance, we further design a Kalman filter to combine with the neuro-based tracker. The instrument can be detected more robustly despite of deformation of the instrument image during surgery. A laparoscopic video has been tested to verify the developed methods. Experimental results show that two instruments can be distinguished and tracked simultaneously in the surgical video.