Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
Abstract
We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process. We first learn a deep generative model of the unlabeled data using the fast, greedy algorithm introduced by Hinton et.al. If the data is high-dimensional and highly-structured, a Gaussian kernel applied to the top layer of features in the DBN works much better than a similar kernel applied to the raw input. Performance at both regression and classification can then be further improved by using backpropagation through the DBN to discriminatively fine-tune the covariance kernel.
Cite
Text
Hinton and Salakhutdinov. "Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes." Neural Information Processing Systems, 2007.Markdown
[Hinton and Salakhutdinov. "Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes." Neural Information Processing Systems, 2007.](https://mlanthology.org/neurips/2007/hinton2007neurips-using/)BibTeX
@inproceedings{hinton2007neurips-using,
title = {{Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes}},
author = {Hinton, Geoffrey E. and Salakhutdinov, Ruslan},
booktitle = {Neural Information Processing Systems},
year = {2007},
pages = {1249-1256},
url = {https://mlanthology.org/neurips/2007/hinton2007neurips-using/}
}