Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure
Abstract
Graphical models are often used “inappropriately,” with approximations in the topology, inference, and prediction. Yet it is still common to train their parameters to approximately maximize training likelihood. We argue that instead, one should seek the parameters that minimize the empirical risk of the entire imperfect system. We show how to locally optimize this risk using back-propagation and stochastic meta-descent. Over a range of synthetic-data problems, compared to the usual practice of choosing approximate MAP parameters, our approach significantly reduces loss on test data, sometimes by an order of magnitude.
Cite
Text
Stoyanov et al. "Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure." Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011.Markdown
[Stoyanov et al. "Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure." Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011.](https://mlanthology.org/aistats/2011/stoyanov2011aistats-empirical/)BibTeX
@inproceedings{stoyanov2011aistats-empirical,
title = {{Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure}},
author = {Stoyanov, Veselin and Ropson, Alexander and Eisner, Jason},
booktitle = {Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics},
year = {2011},
pages = {725-733},
volume = {15},
url = {https://mlanthology.org/aistats/2011/stoyanov2011aistats-empirical/}
}