I See What You Say (ISWYS): Arabic lip reading system

The ability of communicating easily with everyone is a blessing people with hearing impairment do not have. They completely rely on their vision around healthy individuals to difficultly read lips. This paper proposes a solution for this problem, ISWYS (I See What You Say) is a research-oriented speech recognition system for the Arabic language that interprets lips movements into readable text. It is accomplished by analyzing a video of lips movements that resemble utterances, and then converting it to readable characters using video analysis and motion estimation. Our algorithm involves dividing the video into n number of frames to generate n-1 image frame which is produced by taking the difference between consecutive frames. Then, video features are extracted to be used by our error function which provided recognition of approximately 70%.