Title :
A System to Interact with CAVE Applications Using Hand Gesture Recognition from Depth Data
Author :
Leite, Diego Augusto T. Q. ; Duarte, Julio Cesar ; Oliveira, Jauvane C. ; De Almeida Thomaz, Victor ; Giraldi, Gilson Antonio
Author_Institution :
Inst. Mil. de Eng., Rio de Janeiro, Brazil
Abstract :
Human Computer Interaction (HCI) is a fundamental issue for virtual reality environments due to the need for natural approaches and comfortable devices. Such goals can be achieved using hand gestures to interact with the virtual reality engine. This paper presents a real-time system based on hand gesture recognition (HGR) for interaction with CAVE applications. The whole pipeline can be roughly divided into four steps: segmentation, feature extraction for bag-of-features construction, classification through multiclass support vector machine (SVM), generation of commands to control the application. We build a grammar based on the hand gesture classes to convert the classification results in control commands for an application running in a CAVE. The input is the depth stream data acquired from a Kinect device. The hand gesture recognition and command generation/execution approaches compose a client-server plug in that is part of a CAVE system implemented based on the Instant Reality architecture and the X3D standard. The results show that the implemented plug in is a promising solution. We achieve suitable recognition accuracy and efficient object manipulation in a virtual room representing a surgical environment visualized in the CAVE.
Keywords :
feature extraction; gesture recognition; image classification; image segmentation; image sensors; interactive devices; support vector machines; virtual reality; CAVE application; HCI; HGR; Instant Reality architecture; Kinect device; SVM; X3D standard; bag-of-features construction; classification step; client-server plug in; command execution approach; command generation approach; commands generation step; depth data; depth stream data; feature extraction step; hand gesture recognition; human computer interaction; multiclass support vector machine; object manipulation; segmentation step; surgical environment; virtual reality engine; virtual reality environment; virtual room; Cameras; Feature extraction; Gesture recognition; Hardware; Support vector machines; Three-dimensional displays; Training; CAVE; Computer Vision; Hand-Gesture Recognition;
Conference_Titel :
Virtual and Augmented Reality (SVR), 2014 XVI Symposium on
Conference_Location :
Piata Salvador
DOI :
10.1109/SVR.2014.13