Generating Accurate and Diverse Members of a Neural-Network Ensemble

Abstract

Neural-network ensembles have been shown to be very accurate classification techniques. Previous work has shown that an effec(cid:173) tive ensemble should consist of networks that are not only highly correct, but ones that make their errors on different parts of the input space as well. Most existing techniques, however, only in(cid:173) directly address the problem of creating such a set of networks. In this paper we present a technique called ADDEMUP that uses genetic algorithms to directly search for an accurate and diverse set of trained networks. ADDEMUP works by first creating an ini(cid:173) tial population, then uses genetic operators to continually create new networks, keeping the set of networks that are as accurate as possible while disagreeing with each other as much as possible. Ex(cid:173) periments on three DNA problems show that ADDEMUP is able to generate a set of trained networks that is more accurate than sev(cid:173) eral existing approaches. Experiments also show that ADDEMUP is able to effectively incorporate prior knowledge, if available, to improve the quality of its ensemble.