Sequential Labeling with Online Deep Learning: Exploring Model Initialization
Abstract
In this paper, we leverage both deep learning and conditional random fields (CRFs) for sequential labeling. More specifically, we explore parameter initialization and randomization in deep CRFs and train the whole model in a simple but effective way. In particular, we pretrain the deep structure with greedy layer-wise restricted Boltzmann machines (RBMs), followed with an independent label learning step. Finally, we re-randomize the top layer weight and update the whole model with an online learning algorithm – a mixture of perceptron training and stochastic gradient descent to estimate model parameters. We test our model on different challenge tasks, and show that this simple learning algorithm yields the state of the art results. The data and software related to this paper are available at https://github.com/ganggit/deepCRFs .
Cite
Text
Chen et al. "Sequential Labeling with Online Deep Learning: Exploring Model Initialization." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2016. doi:10.1007/978-3-319-46227-1_48Markdown
[Chen et al. "Sequential Labeling with Online Deep Learning: Exploring Model Initialization." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2016.](https://mlanthology.org/ecmlpkdd/2016/chen2016ecmlpkdd-sequential/) doi:10.1007/978-3-319-46227-1_48BibTeX
@inproceedings{chen2016ecmlpkdd-sequential,
title = {{Sequential Labeling with Online Deep Learning: Exploring Model Initialization}},
author = {Chen, Gang and Xu, Ran and Srihari, Sargur N.},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2016},
pages = {772-788},
doi = {10.1007/978-3-319-46227-1_48},
url = {https://mlanthology.org/ecmlpkdd/2016/chen2016ecmlpkdd-sequential/}
}