Human workflow analysis using 3D occupancy grid hand tracking in a human-robot collaboration scenario

In this work, we present a Hidden Markov Model (HMM) based workflow analysis of an assembly task jointly performed by a human and an assistive robotic system. In an experiment subjects had to assemble a tower by combining six cubes with several bolts for their own without the influence of a robot or any other technical device. To estimate the current action of the human, we have trained composite HMMs. After the successful evaluation on disjunct experimental data sets, the models are transferred to the assistive robotic system JAHIR, where the same assembly tasks was executed. A new 3D occupancy grid approach was used to determine the hand positions of the worker. The positions were then used to compute the inputs of the analysis HMMs. The workflow of the right hand could be recognized with an accuracy of 92:26% which is nearly as good as the recognition rate of reference experiments.