Is Nonparametric Learning Practical in Very High Dimensional Spaces?

Abstract

Many of the challenges faced by the field of Computational Intelligence in building intelligent agents, involve determining mappings between numerous and varied sensor inputs and complex and flexible action sequences. In applying nonparametric learning techniques to such problems we must therefore ask: "Is nonparametric learning practical in very high dimensional spaces?" Contemporary wisdom states that variable selection and a "greedy" choice of appropriate functional structures are essential ingredients for nonparametric learning algorithms. However, neither of these strategies is practical when learning problems have thousands of input variables, and tens of thousands of learning examples. We conclude that such nonparametric learning is practical by using a methodology which does not use either of these techniques. We propose a simple nonparametric learning algorithm to support our conclusion. The algorithm is evaluated first on 10 well known regression data sets, where it is shown ...