The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks

To date, the preponderance of techniques for eliciting the knowledge embedded in trained artificial neural networks (ANN´s) has focused primarily on extracting rule-based explanations from feedforward ANN´s. The ADT taxonomy for categorizing such techniques was proposed in 1995 to provide a basis for the systematic comparison of the different approaches. This paper shows that not only is this taxonomy applicable to a cross section of current techniques for extracting rules from trained feedforward ANN´s but also how the taxonomy can be adapted and extended to embrace a broader range of ANN types (e,g., recurrent neural networks) and explanation structures. In addition we identify some of the key research questions in extracting the knowledge embedded within ANN´s including the need for the formulation of a consistent theoretical basis for what has been, until recently, a disparate collection of empirical results