Autonomous and stable tracking of endoscope instrument tools with monocular camera

With an autonomous tool tracking system, surgeons do not need to handle the endoscope during surgery. This paper aims to develop an image tracking system with a stable view for minimally invasive surgery (MIS) by using monocular endoscope. In order to provide a stable view for the surgery, we propose to set a buffer zone in the center of image frame for the robotic camera holder to track the endoscope instruments. If endoscope instruments are inside the buffer zone, the endoscope robot will keep still. However, if one or both of them move away from the buffer zone, the robot will start to track the instrument. Furthermore, by calculating the distance between two instruments in the image plane, the robotic camera holder can track the depth to the instruments. The robot will therefore zoom in if the distance is too short. A safe distance from the camera lens to instrument tools can therefore be maintained during the image tracking. Preliminary experimental results show that the proposed design achieves satisfactory tracking accuracy in real time and provides a stable view simultaneously.