Discovering Homogeneous Regions in Spatial Data Through Competition

Abstract

If all features causing heterogeneity were observed, a mixture of experts approach (Jacobs et al., 1991) is likely to be superior to using a single model. When unobserved or very noisy spatial features are the cause for the heterogeneity, the observed feature spaces of homogeneous subsets can highly overlap, leading to a biased global model or biased mixture of experts. Our goal is to allow more accurate predictions in such situations. Here, a supervised machine learning algorithm for the analysis of heterogeneous spatial data is proposed. It is based on partitioning the data set into more homogeneous regions by competition of regression models (linear or nonlinear). The algorithm starts from learning a global model, and adds new models into the competition until each model becomes specialized for one of the regions. The competition convergence is proven theoretically. Also, the influence of filtering the competing models residuals for improving convergence speed and accuracy is discussed. A number of experiments on artificial and real-life spatial data are performed to validate some aspects of the algorithm and to illustrate its potential applications. The obtained results provide strong evidence that homogeneous regions can be identified with high accuracy by using the proposed approach even when their observed feature spaces highly overlap. An assumption of data independence valid for most of standard machine learning data sets is often unrealistic for spatial variables, whose dependence is strongly tied to a location, where observations spatially close to each other are more likely to be similar than observations widely separated in space. As a consequence, errors of spatial prediction models are also spatially correlated (Cressie, 1993). The method proposed here incorporates knowledge of spatial correlation for more accurate partitioning of heterogeneous spatial data sets. The new partitioning algorithm is based on three important mechanisms: (a) competition among learning models for spatial data points, (b) averaging errors of each competing model over neighboring data points, and (c) an incremental introduction of additional models into the competition when needed.