Classification Trees with Neural Network Feature Extraction

Abstract

The use of small multilayer nets at the decision nodes of a binary classification tree to extract nonlinear features is proposed. This approach exploits the power of tree classifiers to use appropriate local features at the different levels and nodes of the tree. The nets are trained and the tree is grown using a gradient-type learning algorithm in conjunction with a heuristic class aggregation algorithm. The method improves on standard classification tree design methods in that it generally produces trees with lower error rates and fewer nodes. It also provides a structured approach to neural network classifier design which reduces the problem associated with training large unstructured nets, and transfers the problem of selecting the size of the net to the simpler problem of finding the right size tree. Example concern waveform and handwritten character recognition.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>