Pixelwise object class segmentation based on synthetic data using an optimized training strategy

In this paper we present an approach for low-level body part segmentation based on RGB-D data. The RGB-D sensor is thereby placed at the ceiling and observes a shared workspace for human-robot collaboration in the industrial domain. The pixelwise information about certain body parts of the human worker is used by a cognitive system for the optimization of interaction and collaboration processes. In this context, for rational decision making and planning, the pixelwise predictions must be reliable despite the high variability of the appearance of the human worker. In our approach we treat the problem as a pixelwise classification task, where we train a random decision forest classifier on the information contained in depth frames produced by a synthetic representation of the human body and the ceiling sensor, in a virtual environment. As shown in similar approaches, the samples used for training need to cover a broad spectrum of the geometrical characteristics of the human, and possible transformations of the body in the scene. In order to reduce the number of training samples and the complexity of the classifier training, we therefore apply an elaborated and coupled strategy for randomized training data sampling and feature extraction. This allows us to reduce the training set size and training time, by decreasing the dimensionality of the sampling parameter space. In order to keep the creation of synthetic training samples and real-world ground truth data simple, we use a highly reduced virtual representation of the human body, in combination with KINECT skeleton tracking data from a calibrated multi-sensor setup. The optimized training and simplified sample creation allows us to deploy standard hardware for the realization of the presented approach, while yielding a reliable segmentation in real-time, and high performance scores in the evaluation.