DocumentCode :
3543063
Title :
Combining depth image and skeleton data from Kinect for recognizing words in the sign system for Indonesian language (SIBI [Sistem Isyarat Bahasa Indonesia])
Author :
Rakun, Erdefi ; Andriani, Made ; Wiprayoga, I. Wayan ; Danniswara, Ken ; Tjandra, Andros
Author_Institution :
Fac. of Comput. Sci., Univ. Indonesia, Depok, Indonesia
fYear :
2013
fDate :
28-29 Sept. 2013
Firstpage :
387
Lastpage :
392
Abstract :
The Sign System for Indonesian Language (SIBI) is a rather complex sign language. It has four components that distinguish the meaning of the sign language and it follows the syntax and the grammar of the Indonesian language. This paper proposes a model for recognizing the SIBI words by using Microsoft Kinect as the input sensor. This model is a part of automatic translation from SIBI to text. The features for each word are extracted from skeleton and color-depth data produced by Kinect. Skeleton data features indicate the angle between human joints and Cartesian axes. Color images are transformed to gray-scale and their features are extracted by using Discrete Cosine Transform (DCT) with Cross Correlation (CC) operation. The image´s depth features are extracted by running MATLAB regionprops function to get its region properties. The Generalized Learning Vector Quantization (GLVQ) and Random Forest (RF) training algorithm from WEKA data mining tools are used as the classifier of the model. Several experiments with different scenarios have shown that the highest accuracy (96,67%) is obtained by using 30 frames for skeleton combined with 20 frames for region properties image classified by Random Forest.
Keywords :
data mining; decision trees; discrete cosine transforms; image classification; image colour analysis; image thinning; language translation; learning (artificial intelligence); natural language processing; sign language recognition; vector quantisation; Cartesian axis; DCT; GLVQ; Indonesian language grammar; Indonesian language syntax; MATLAB regionprops function; Microsoft Kinect; SIBI word recognition; Sistem Isyarat Bahasa Indonesia; WEKA data mining tools; automatic translation; color image transformation; color-depth data; cross correlation operation; discrete cosine transform; generalized learning vector quantization; gray-scale image; human joints; image classification; image depth feature; random forest training algorithm; region properties; sign language meaning; sign system; skeleton data feature; word feature extraction; Accuracy; Assistive technology; Feature extraction; Joints; Mathematical model; Shape;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Advanced Computer Science and Information Systems (ICACSIS), 2013 International Conference on
Conference_Location :
Bali
Type :
conf
DOI :
10.1109/ICACSIS.2013.6761606
Filename :
6761606
Link To Document :
بازگشت