Dimension Reduction Techniques for Training Polynomial Networks

Abstract

We propose two novel methods for reducing dimension in training polynomial networks. We consider the class of polynomial networks whose output is the weighted sum of a basis of monomials. Our first method for dimension reduction eliminates redundancy in the training process. Using an implicit matrix structure, we derive iterative methods that converge quickly. A second method for dimension reduction involves a novel application of random dimension reduction to "feature space." The combination of these algorithms produces a method for training polynomial networks on large data sets with decreased computation over traditional methods and model complexity reduction and control. 1. Introduction We consider polynomial networks of the following type. The inputs, x 1 , ..., xM , to the network are combined with multipliers to form a vector of basis functions p(x); for example, for two inputs x 1 and x 2 and a second degree network, we obtain p(x) = 1 x 1 x 2 x 2 1 x 1 x...