Constructing Heterogeneous Committees Using Input Feature Grouping: Application to Economic Forecasting

Abstract

The committee approach has been proposed for reducing model uncertainty and improving generalization performance. The ad(cid:173) vantage of committees depends on (1) the performance of individ(cid:173) ual members and (2) the correlational structure of errors between members. This paper presents an input grouping technique for de(cid:173) signing a heterogeneous committee. With this technique, all input variables are first grouped based on their mutual information. Sta(cid:173) tistically similar variables are assigned to the same group. Each member's input set is then formed by input variables extracted from different groups. Our designed committees have less error cor(cid:173) relation between its members, since each member observes different input variable combinations. The individual member's feature sets contain less redundant information, because highly correlated vari(cid:173) ables will not be combined together. The member feature sets con(cid:173) tain almost complete information, since each set contains a feature from each information group. An empirical study for a noisy and nonstationary economic forecasting problem shows that commit(cid:173) tees constructed by our proposed technique outperform committees formed using several existing techniques.