Noisy SMS text normalization model

Today digital media such as social networks, chat rooms, and forums have gained much importance in human life for information sharing. Users will share their knowledge and emotions in their own languages. This will create a novel syntax to communicate their messages with as much as pithiness as possible. Noisy text is characterized by unusual forms such as abbreviations, phonetic translations, short forms etc. This led to the emergence of text normalization. Cleaning the noisy text has become an important factor for adequate development and deployment of NLP (Natural Language Processing) services such as text-to-speech and automatic translation. In this paper we introduce a channel based normalization model for cleaning the noisy texts. The normalization is based on the types of distortion such as grapheme distortion, abbreviation and phoneme distortion. The model will explore the type of distortion occurred in the noisy word and replace it by using the different channel list. Precursory evaluation shows that the channel model will normalize the noisy word to their standard peer with 96.43 % accuracy.