Structured Prediction of Sequences and Trees Using Infinite Contexts

Ehsan Shareghi, Gholamreza Haffari, Trevor Cohn, Ann E. Nicholson

ECML-PKDD 2015 pp. 373-389

doi:10.1007/978-3-319-23525-7_23 /ecmlpkdd/2015/shareghi2015ecmlpkdd-structured/

Abstract

Linguistic structures exhibit a rich array of global phenomena, however commonly used Markov models are unable to adequately describe these phenomena due to their strong locality assumptions. We propose a novel hierarchical model for structured prediction over sequences and trees which exploits global context by conditioning each generation decision on an unbounded context of prior decisions. This builds on the success of Markov models but without imposing a fixed bound in order to better represent global phenomena. To facilitate learning of this large and unbounded model, we use a hierarchical Pitman-Yor process prior which provides a recursive form of smoothing. We propose prediction algorithms based on A* and Markov Chain Monte Carlo sampling. Empirical results demonstrate the potential of our model compared to baseline finite-context Markov models on three tasks: morphological parsing, syntactic parsing and part-of-speech tagging.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Shareghi et al. "Structured Prediction of Sequences and Trees Using Infinite Contexts." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2015. doi:10.1007/978-3-319-23525-7_23

Markdown

[Shareghi et al. "Structured Prediction of Sequences and Trees Using Infinite Contexts." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2015.](https://mlanthology.org/ecmlpkdd/2015/shareghi2015ecmlpkdd-structured/) doi:10.1007/978-3-319-23525-7_23

BibTeX

@inproceedings{shareghi2015ecmlpkdd-structured,
  title     = {{Structured Prediction of Sequences and Trees Using Infinite Contexts}},
  author    = {Shareghi, Ehsan and Haffari, Gholamreza and Cohn, Trevor and Nicholson, Ann E.},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2015},
  pages     = {373-389},
  doi       = {10.1007/978-3-319-23525-7_23},
  url       = {https://mlanthology.org/ecmlpkdd/2015/shareghi2015ecmlpkdd-structured/}
}