Endoscopic vision based tracking of multiple surgical instruments in robot-assisted surgery

Robot-assisted cardiac surgeries are effective for operations in limited spaces. Enhancement in safety functions based on automatic tracking of surgical instrument position to prevent inadvertent harmful events as cardiac tissue perforation and instrument collisions can be meaningful augmentation to current robot surgery system. A vision based instrument tracking scheme as a core algorithm to implement such assistive functions has been developed in this study. Through a chain of computer vision techniques, including classification of metallic properties using k-means clustering and instrument movement tracking using similarity measure, Euclidean distance calculation, and Kalman filter algorithm, an automatic tracking scheme has been proposed and the implemented system showed satisfactory performance in tests using real robot-assisted surgery videos. Trajectory comparisons of automatically detected data and ground truth data obtained by manually locating the center of mass of each instrument were used to quantitatively validate the system. The developed algorithm could provide valuable information to clinicians for safer operation.