Natural interactive 3D medical image viewer based on finger and arm gestures

In this research, we propose a natural interaction on 3D medical image viewer software. The goal is to replace the windows, icon, mouse and pointer (WIMP) paradigm which might not be convenient in an operating room as sterilization becomes an issue. The Kinect sensor was used as a controller-free input. The fingertip tracking algorithm was implemented and, combined with existing skeletons from the Kinect sensor. Both finger and arm gestures are used for controlling the 3D medical viewer software. Suitable gestures were designed. We tested our proposed technique in terms of accuracy and performance using 10 subjects. The average accuracy for gesture classification is 93%, and the total time required to cover all controls of 3D medical viewer is about 2.25 minutes on average. The results also show that the proposed technique is applicable for controlling 3D medical image viewer software without the WIMP system.