A Comparison of Two MCMC Algorithms for Hierarchical Mixture Models

Abstract

Mixture models form an important class of models for unsupervised learning, allowing data points to be assigned labels based on their values. However, standard mixture models procedures do not deal well with rare components. For example, pause times in student essays have di↵erent lengths depend-ing on what cognitive processes a student engages in during the pause. However, in-stances of student planning (and hence very long pauses) are rare, and thus it is dif-ficult to estimate those parameters from a single student’s essays. A hierarchical mix-ture model eliminates some of those prob-lems, by pooling data across several of the higher level units (in the example students) to estimate parameters of the mixture com-ponents. One way to estimate the parame-ters of a hierarchical mixture model is to use MCMC. But these models have several issues such as non-identifiability under label switch-ing that make them dicult to estimate just using o↵-the-shelf MCMC tools. This paper looks at the steps necessary to estimate these models using two popular MCMC packages: JAGS (random walk Metropolis algorithm) and Stan (Hamiltonian Monte Carlo). JAGS, Stan and R code to estimate the models and model fit statistics are published along with the paper.