Repairing Faulty Mixture Models Using Density Estimation

Abstract

Previous work in mixture model clustering has focused primarily on the issue of model selection. Model scoring functions (including penalized likelihood and Bayesian approximations) can guide a search of the model parameter and structure space. Relatively little research has addressed the issue of how to move through this space. Local optimization techniques, such as expectation maximization, solve only part of the problem; we still need to move between different local optima. The traditional approach, restarting the search from different random configurations, is inefficient. We describe a more directed and controlled way of moving between local maxima. Using multi-resolution kd- trees for fast density estimation, we search by modifying models within regions where they fail to predict the datapoint density. We compare this algorithm with a canonical clustering method, finding favorable results on a variety of large, low-dimensional datasets. 1.