Unsupervised Bayesian Part of Speech Inference with Particle Gibbs
Abstract
As linguistic models incorporate more subtle nuances of language and its structure, standard inference techniques can fall behind. These models are often tightly coupled such that they defy clever dynamic programming tricks. Here we demonstrate that Sequential Monte Carlo approaches, i.e. particle filters, are well suited to approximating such models. We implement two particle filters, which jointly sample either sentences or word types, and incorporate them into a Particle Gibbs sampler for Bayesian inference of syntactic part-of-speech categories. We analyze the behavior of the samplers and compare them to an exact block sentence sampler, a local sampler, and an existing heuristic word type sampler. We also explore the benefits of mixing Particle Gibbs and standard samplers.
Cite
Text
Dubbin and Blunsom. "Unsupervised Bayesian Part of Speech Inference with Particle Gibbs." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2012. doi:10.1007/978-3-642-33460-3_54Markdown
[Dubbin and Blunsom. "Unsupervised Bayesian Part of Speech Inference with Particle Gibbs." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2012.](https://mlanthology.org/ecmlpkdd/2012/dubbin2012ecmlpkdd-unsupervised/) doi:10.1007/978-3-642-33460-3_54BibTeX
@inproceedings{dubbin2012ecmlpkdd-unsupervised,
title = {{Unsupervised Bayesian Part of Speech Inference with Particle Gibbs}},
author = {Dubbin, Gregory and Blunsom, Phil},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2012},
pages = {760-773},
doi = {10.1007/978-3-642-33460-3_54},
url = {https://mlanthology.org/ecmlpkdd/2012/dubbin2012ecmlpkdd-unsupervised/}
}