Cognitive face analysis system for future interactive TV

The future interactive TV will automatically provide user-personalized services for each viewer. For the automatic user-personalized services, the interactive TV should recognize viewers and even their emotions or preferences. In this paper, we introduce a novel architecture of the future interactive TV and propose a real-time face analysis system that can detect and recognize human faces and even their expressions, and therefore understand their internal emotional states. The proposed face analysis system consists of three image processing modules: face detection, face recognition, and facial expression recognition. Face detection is employed to detect and track a number of people watching the TV program. Face recognition and facial expression recognition are employed to identify specific TV viewers and recognize their internal emotional states, which is necessary in order to enable personalized user interfaces and services. For robust real-time face analysis system, we present the Ada-LDA learning algorithm based on simple and efficient MspLBP features, which is suitable for multi-class pattern classification. Extensive experimental results show that the proposed face analysis system provides real-time performance with high recognition rates. It can operate at over 15 frames per second.