Semi-optimal hierarchical regression models and ANNs

A hierarchical modular model is comprised of a set of specialized models (committee machine) that are constructed in hierarchical (tree) fashion and each of which is responsible for a particular region of input space. Many algorithms in this class, for example M5 model tree, are greedy and hence far from being optimal. An algorithmic framework leading to building more accurate optimal and semi-optimal trees is proposed. Its particular implementation for regression problems, M5opt algorithm, constructs model trees that are more accurate than those obtained by the greedy approach of M5, and in a number of cases more accurate than ANNs.