Machine learning of engineering diagnostic knowledge from unstructured verbatim text descriptions

This paper presents our research in text mining for discovering important engineering fault diagnostic knowledge from unstructured and verbatim text descriptions. In particular we focus on developing machine learning algorithms for detecting documents that contain descriptions of systematic failures and root causes to the faults. We developed a machine algorithm based on entropy analysis to extract an A-word list, a list of words that are important to characterize the documents of interests, a vector space model to represent features of important documents, and a constraint based k-means clustering algorithm to generate high purity clusters for use in detecting important documents. We applied the algorithms to automotive diagnostic text data, which are unstructured and verbatim descriptions by customers and technicians that contain many typos and self-invented terms. We were able to reduce a list of 2183 words to a list of 137 important words. The classification system generated by these machine learning algorithms showed high recall and accuracy in detecting important diagnostic descriptions.