Cascaded Data Mining Methods for Text Understanding, with medical case study

Substantial electronically stored textual data such as clinical narratives reports often need to be retrieved to find relevant information for clinical and research purposes. The context of negation, a negative finding, is of special importance, since many of the most frequently described findings are such. Hence, when searching free-text narratives for patients with a certain medical condition, if negation is not taken into account, many of the documents retrieved were irrelevant. We present a new cascaded pattern learning method for automatic identification of negative context in clinical narratives reports. Studying the training corpuses, the classification errors and patterns selected by the classifier, we noticed that it is possible to create a more powerful ensemble structure than the structure obtained from general-purpose ensemble method (such as Adaboost). We compare the new algorithm to previous methods proposed for the same task of similar medical narratives, and show its advantages: accuracy improvement compared to other machine learning methods, and much faster than manual knowledge engineering techniques with matching accuracy