Successive Refinement and Coarsening of the Information Bottleneck

Abstract

We study two central aspects of information processing in cognitive systems: one is the ability to incorporate fresh information to already learnt models; the other is the ``trickling'' of information through the many layers of a cognitive processing pipeline. We investigate the extent to which these specific structures of cognitive processing impact their informational optimal limits. To do so, we present mathematical characterisations and low-dimensional numerical examples, which explore formal properties of the Information Bottleneck method: namely, how it relates to successive refinement, and successive coarsening of information.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Charvin et al. "Successive Refinement and Coarsening of the Information Bottleneck." NeurIPS 2022 Workshops: InfoCog, 2022.

Markdown

[Charvin et al. "Successive Refinement and Coarsening of the Information Bottleneck." NeurIPS 2022 Workshops: InfoCog, 2022.](https://mlanthology.org/neuripsw/2022/charvin2022neuripsw-successive/)

BibTeX

@inproceedings{charvin2022neuripsw-successive,
  title     = {{Successive Refinement and Coarsening of the Information Bottleneck}},
  author    = {Charvin, Hippolyte and Polani, Daniel and Volpi, Nicola Catenacci},
  booktitle = {NeurIPS 2022 Workshops: InfoCog},
  year      = {2022},
  url       = {https://mlanthology.org/neuripsw/2022/charvin2022neuripsw-successive/}
}