Predictive Coding, Variational Autoencoders, and Biological Connections

Abstract

Predictive coding, within theoretical neuroscience, and variational autoencoders, within machine learning, both involve latent Gaussian models and variational inference. While these areas share a common origin, they have evolved largely independently. We outline connections and contrasts between these areas, using their relationships to identify new parallels between machine learning and neuroscience. We then discuss specific frontiers at this intersection: backpropagation, normalizing flows, and attention, with mutual benefits for both fields.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Marino. "Predictive Coding, Variational Autoencoders, and Biological Connections." NeurIPS 2019 Workshops: Neuro_AI, 2019.

Markdown

[Marino. "Predictive Coding, Variational Autoencoders, and Biological Connections." NeurIPS 2019 Workshops: Neuro_AI, 2019.](https://mlanthology.org/neuripsw/2019/marino2019neuripsw-predictive/)

BibTeX

@inproceedings{marino2019neuripsw-predictive,
  title     = {{Predictive Coding, Variational Autoencoders, and Biological Connections}},
  author    = {Marino, Joseph},
  booktitle = {NeurIPS 2019 Workshops: Neuro_AI},
  year      = {2019},
  url       = {https://mlanthology.org/neuripsw/2019/marino2019neuripsw-predictive/}
}