Determination of linguistic differences and statistical analysis of large corpora of Indian languages

This paper presents statistical analysis of large corpora for three Indian languages i.e. Hindi, Punjabi and Nepali. The main objective of this study is to analyze the statistical features of these three languages that may be helpful for finding distinction among these languages. Detailed statistical analyses have been done to compute the information about entropy, perplexity, word length, coverage analysis, vocabulary growth rate etc. Most frequently occurring words have been extracted from the text corpus of three languages. Based on statistical features a comparative analysis has been done to find the similarities and differences among these languages.