Title :
Learning optimization for decision tree classification of non-categorical data with information gain impurity criterion
Author :
Sofeikov, K.I. ; Tyukin, I.Yu. ; Gorban, A.N. ; Mirkes, E.M. ; Prokhorov, Danil V. ; Romanenko, I.V.
Author_Institution :
Dept. of Math., Univ. of Leicester, Leicester, UK
Abstract :
We consider the problem of construction of decision trees in cases when data is non-categorical and is inherently high-dimensional. Using conventional tree growing algorithms that either rely on univariate splits or employ direct search methods for determining multivariate splitting conditions is computationally prohibitive. On the other hand application of standard optimization methods for finding locally optimal splitting conditions is obstructed by abundance of local minima and discontinuities of classical goodness functions such as e.g. information gain or Gini impurity. In order to avoid this limitation a method to generate smoothed replacement for measuring impurity of splits is proposed. This enables to use vast number of efficient optimization techniques for finding locally optimal splits and, at the same time, decreases the number of local minima. The approach is illustrated with examples.
Keywords :
decision trees; learning (artificial intelligence); optimisation; pattern classification; Gini impurity; classical goodness functions; conventional tree growing algorithms; direct search methods; high-dimensional data; information gain impurity criterion; learning optimization; multivariate splitting conditions; noncategorical data decision tree classification; split impurity; univariate splits; Decision trees; Electronic mail; Entropy; Hafnium; Impurities; Optimization; Standards;
Conference_Titel :
Neural Networks (IJCNN), 2014 International Joint Conference on
Conference_Location :
Beijing
Print_ISBN :
978-1-4799-6627-1
DOI :
10.1109/IJCNN.2014.6889842