Challenges and techniques for mining real clinical data

Regression analysis and statistical hypothesis testing are commonly used for association and classification of clinical data sets in medical studies. Although such traditional techniques are wildly used, they have several shortcomings. For example, when analyzing datasets with a large number of temporal attributes, domain experts often miss important associative attributes in regression analysis because of the large number of correlated attributes. On the other hand, for rare occurring diseases or operations, the number of documented observed cases is usually small, and hypothesis testing becomes ineffective for such analysis due to insufficient statistical significance. We shall present two such case studies to showcase how data mining techniques [1-7] can be used to remedy such shortcomings.