Evaluating preprocessing by turing machine in text categorization

By developing the World Wide Web, text categorization becomes a key way to deal with a large number of data and organize them. Automatic text categorization has three steps: preprocessing, extracting relevant features and categorization documents into specified categories. In this article, we propose a new preprocessing method by Turing Machine. All of four steps in preprocessing such as sentence segmentation, tokenization, stop word removal and word stemming are done by Turing Machine. The support vector machine model on the Reuters and PAGOD dataset is used to present importance of preprocessing by Turing Machine. We used from term weighting, feature subset selection and feature reduction techniques to find the best document representation. Experiments show that our proposed method is more accurate than other methods.