A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning
Abstract
We describe a single convolutional neural network architecture that given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic roles, semantically similar words and the likelihood that the sentence makes sense (grammatically and semantically) using a language model. The entire network is trained jointly on all these tasks using weight-sharing, an instance of multitask learning. All the tasks use labeled data except the language model which is learnt from unlabeled text and represents a novel way of performing semi-supervised learning for the shared tasks. We show how both multitask learning and semi-supervised learning improve the generalization of the shared tasks, resulting in a learnt model with state-of-the-art performance.
Cite
Text
Collobert and Weston. "A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning." International Conference on Machine Learning, 2008. doi:10.1145/1390156.1390177Markdown
[Collobert and Weston. "A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning." International Conference on Machine Learning, 2008.](https://mlanthology.org/icml/2008/collobert2008icml-unified/) doi:10.1145/1390156.1390177BibTeX
@inproceedings{collobert2008icml-unified,
title = {{A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning}},
author = {Collobert, Ronan and Weston, Jason},
booktitle = {International Conference on Machine Learning},
year = {2008},
pages = {160-167},
doi = {10.1145/1390156.1390177},
url = {https://mlanthology.org/icml/2008/collobert2008icml-unified/}
}