Effective hand segmentation and gesture recognition for browsing web pages on a large screen

Modern digital family technology enables people surf the Internet and watch videos via a large screen. This paper proposes an effective scheme for using hand gestures rather than the common remote controllers to browse the web pages on a large TV screen. The proposed scheme models four gesture modes: mouse mode, scroll mode, zoom mode and input mode to help the user browse Web pages naturally and comfortably. Then we combine RGB, depth, motion information and face detection to achieve accurate and real-time hand segmentation and gesture recognition for enabling the four gesture modes. The experiments show the proposed scheme works well in various illumination environments and complicated backgrounds with multiple moving humans. The recognition accuracy of hand shapes in the proposed scheme arrives at 98.50%, and the successful rate for visual digits input reaches 89.00%. Furthermore, the frame rate of the hand-gesture detection and recognition is about 18 fps. Thus the scheme is accurate, real-time and natural.