Robust fall detection with an assistive humanoid robot

Summary form only given. In this video we introduce a robot assistant that monitors a person in a household environment to promptly detect fall events. In contrast to the use of a fixed sensor, the humanoid robot will track and keep the moving person in the scene while performing daily activities. For this purpose, we extended the humanoid Nao¹ with a depth sensor² attached to its head. The tracking framework implemented with OpenNI³ segments and tracks the person´s position and body posture. We use a learning neural framework for processing the extracted body features and detecting abnormal behaviors, e.g. a fall event [1]. The neural architecture consists of a hierarchy of self-organizing neural networks for attenuating noise caused by tracking errors and detecting fall events from video stream in real time. The tracking application, the neural framework, and the humanoid actuators communicate over Robot Operating System (ROS)⁴. We use communication over the ROS network implemented with publisher-subscriber nodes. When a fall event is detected, Nao will approach the person and ask whether assistance is needed. In any case, Nao will take a picture of the scene that can be sent to the caregiver or a relative for further human evaluation and agile intervention. The combination of this sensor technology with our neural network approach allows to tailor the robust detection of falls independently from the background surroundings and in the presence of noise (tracking errors and occlusions) introduced by a real-world scenario. The video shows experiments run in a home-like environment.