A telephonic lipreading device for the hearing impaired

The authors discuss the design of a pattern recognition system to extract appropriate lip-shapes from a speech waveform where no other cues are available. The principle of the system depends on there being an intimate relationship between lip-shapes and the corresponding state of the vocal tract. Because of this association at the articulatory level, it is believed that many of the problems associated with speech recognition systems are avoided in this instance. Careful consideration of the training procedures has been required, because of the problem of storing large amounts of visual and spoken data. The self organising property of the Kohonen map was seen as a key feature of the system design. In addition, the use of linear prediction in the pre-processor stage ensures that the data presented to the pattern recogniser is representative of those features of the vocal tract relevant to the association with the lip-shape. The design of archetypal lip-shapes has been presented and may be seen as a quantisation of the visual speech domain