Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
Abstract
We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method often significantly outperforms existing methods in terms of the training objective, and performs as well or better than optimally-tuned stochastic gradient methods in terms of test error.
Cite
Text
Schmidt et al. "Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields." International Conference on Artificial Intelligence and Statistics, 2015.Markdown
[Schmidt et al. "Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields." International Conference on Artificial Intelligence and Statistics, 2015.](https://mlanthology.org/aistats/2015/schmidt2015aistats-non/)BibTeX
@inproceedings{schmidt2015aistats-non,
title = {{Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields}},
author = {Schmidt, Mark and Babanezhad, Reza and Ahmed, Mohamed Osama and Defazio, Aaron and Clifton, Ann and Sarkar, Anoop},
booktitle = {International Conference on Artificial Intelligence and Statistics},
year = {2015},
url = {https://mlanthology.org/aistats/2015/schmidt2015aistats-non/}
}