Online Random Forests Based on CorrFS and CorrBE

Abstract

This paper aims to contribute to the merits of online ensemble learning for classification problems. To this end we induce random forests algorithm into online mode and estimate the importance of variables incrementally based on correlation ranking (CR). We test our method by an "incremental hill climbing" algorithm in which features are greedily added in a "forward" step (FS), and removed in a "backward" step (BE). We resort to an implementation that combine CR with FS and BE. We call this implementation CorrFS and CorrBE respectively. Evaluation based on public UCI databases demonstrates that our method can achieve comparable performance to classifiers constructed from batch training. In addition, the framework allows a fair comparison among other batch mode feature selection approaches such as Gini index, ReliefF and gain ratio.