Combining Motion Sensors and Ultrasonic Hands Tracking for Continuous Activity Recognition in a Maintenance Scenario

We present a novel method for continuous activity recognition based on ultrasonic hand tracking and motion sensors attached to the user´s arms. It builds on previous work in which we have shown such a sensor combination to be effective for isolated recognition in manually segmented data. We describe the hand tracking based segmentation, show how classification is done on both the ultrasonic and the motion data and discuss different classifier fusion methods. The performance of our method is investigated in a large scale experiment in which typical bicycle repair actions are performed by 6 different subjects. The experiment contains a test set with 1008 activities from 21 classes encompassing 115 minutes randomly mixed with 252 minutes of ´NULL´ class. To come as close as possible to a real life continuous scenario we have ensured a diverse and complex ´NULL´ class, diverse and often similar activities, inter person training/testing and an additional data set only for training (299 extra minutes of data). A key result of the paper is that our method can handle the user independent testing (testing on users that were not seen in training) nearly as well as the user dependent case.