Generating Structure of Latent Variable Models for Nested Data

Abstract

Probabilistic latent variable models have been successfully used to capture intrinsic character-istics of various data. However, it is nontrivial to design appropriate models for given data because it requires both machine learning and domain-specific knowledge. In this paper, we focus on data with nested structure and propose a method to automatically generate a latent variable model for the given nested data, with the proposed method, the model structure is adjustable by its structural parameters. Our model can represent a wide class of hierarchical and sequential la-tent variable models including mixture models, latent Dirichlet allocation, hidden Markov mod-els and their combinations in multiple layers of the hierarchy. Even when deeply-nested data are given, where designing a proper model is diffi-cult even for experts, our method generate an ap-propriate model by extracting the essential infor-mation. We present an efficient variational in-ference method for our model based on dynamic programming on the given data structure. We ex-perimentally show that our method generates cor-rect models from artificial datasets and demon-strate that models generated by our method can extract hidden structures of blog and news article datasets. 1