Multi-sensor based segmentation of human manipulation tasks

In this paper we present an overview of a multi-sensor setup designed to record and analyse human in-hand manipulation - tasks consisting of several phases of finger motions following the initial grasp. During the experiments all of the hand, finger, and object positions are recorded, as are the contact forces applied to the manipulated objects. The use of instrumented sensing objects complements the data. The goal is to understand and extract a basic set of finger and hand movement patterns, which can then be combined to perform a complete manipulation task, and which can be transferred to control robotic hands. The segmentation of whole manipulation traces into several phases corresponding to individual basic patterns is the first step towards this goal. Initial analysis and segmentation of two typical manipulation tasks are presented, showing the advantages of the multi-modal analysis.