Depth-based posture recognition by radar and vision fusion for real-time applications

A radar sensor can capture the distance and angle of an object. Mapping the radar distance and angle information to the coordinates of a video frame accelerates the speed of object identification. The distance information is used to calibrate the size of an object to help the recognition. To achieve real-time performance, we use only five center of gravity points (COG) and four feature sets. Two feature sets measure the displacement of the upper and lower body COG in the vertical and horizontal directions. The other two feature sets quantize the upper and lower body angular change rate. The simulation results show that our proposed approach achieve 98.02% to 80.20% recognition rates for various postures and actions in the KTH and ISIR databases.