Study of the word segmentation algorithm based on Hash dictionary mechanism

Machine learning of data analysis and processing in internet allows users to access information quickly and conveniently. As most of the information is text, so that automatic segmentation technology has great significance. The word segmentation dictionary is an important component of the Chinese automatic word segmentation system. The speed of the dictionary loading and query can affect the speed of segmentation system directly. This paper proposes an improved word segmentation mechanism based on double word Hash. The test result shows that the improved word segmentation algorithm enhances the query speed and efficiency of the term matches.