Learning Path Distributions Using Nonequilibrium Diffusion Networks
Abstract
We propose diffusion networks, a type of recurrent neural network with probabilistic dynamics, as models for learning natural signals that are continuous in time and space. We give a formula for the gradient of the log-likelihood of a path with respect to the drift parameters for a diffusion network. This gradient can be used to optimize diffusion networks in the nonequilibrium regime for a wide variety of problems paralleling techniques which have succeeded in engineering fields such as system identification, state estimation and signal filtering. An aspect of this work which is of particu(cid:173) lar interest to computational neuroscience and hardware design is that with a suitable choice of activation function, e.g., quasi-linear sigmoidal, the gradient formula is local in space and time.
Cite
Text
Mineiro et al. "Learning Path Distributions Using Nonequilibrium Diffusion Networks." Neural Information Processing Systems, 1997.Markdown
[Mineiro et al. "Learning Path Distributions Using Nonequilibrium Diffusion Networks." Neural Information Processing Systems, 1997.](https://mlanthology.org/neurips/1997/mineiro1997neurips-learning/)BibTeX
@inproceedings{mineiro1997neurips-learning,
title = {{Learning Path Distributions Using Nonequilibrium Diffusion Networks}},
author = {Mineiro, Paul and Movellan, Javier R. and Williams, Ruth J.},
booktitle = {Neural Information Processing Systems},
year = {1997},
pages = {598-604},
url = {https://mlanthology.org/neurips/1997/mineiro1997neurips-learning/}
}