Modularity in Biologically Inspired Representations Depends on Task Variable Range Independence

Abstract

Artificial and biological neurons sometimes modularise into disjoint groups each encoding a single meaningful variable; at other times they entangle the representation of many variables. Understanding why and when this happens would both help machine learning practitioners build interpretable representations and increase our understanding of neural wetware. In this work, we study optimal neural representations under the biologically inspired constraints of nonnegativity and energy efficiency. We develop a theory of the necessary and sufficient conditions on task structure that induce neural modularisation of task-relevant variables in both linear and partially nonlinear settings. Our theory shows that modularisation is governed not by statistical independence of underlying variables as previously thought, but rather by the independence of the ranges of these variables. We corroborate our theoretical predictions in a variety of empirical studies training feedforward and recurrent neural networks on supervised and unsupervised tasks. Furthermore, we apply these ideas to neuroscience data, providing an explanation of why prefrontal working memory representations sometimes encode different memories in orthogonal subspaces, and sometimes don't, depending on task structure. Lastly, we suggest a suite of surprising settings in which neurons might be or appear mixed selective without requiring complex nonlinear readouts, as in traditional theories. In summary, our theory prescribes precise conditions on when neural activities modularise, providing tools for inducing and elucidating modular representations in machines and brains.