Emergent Representations in Networks Trained with the Forward-Forward Algorithm
Abstract
Backpropagation has been criticised for its lack of biological realism. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise spontaneously into category-specific ensembles exhibiting high sparsity. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Our findings suggest that the learning method used by Forward-Forward may be more biologically plausible than Backpropagation, particularly in terms of the emergent representations it produces.
Cite
Text
Tosato et al. "Emergent Representations in Networks Trained with the Forward-Forward Algorithm." ICML 2024 Workshops: HiLD, 2024.Markdown
[Tosato et al. "Emergent Representations in Networks Trained with the Forward-Forward Algorithm." ICML 2024 Workshops: HiLD, 2024.](https://mlanthology.org/icmlw/2024/tosato2024icmlw-emergent/)BibTeX
@inproceedings{tosato2024icmlw-emergent,
title = {{Emergent Representations in Networks Trained with the Forward-Forward Algorithm}},
author = {Tosato, Niccolo and Basile, Lorenzo and Ballarin, Emanuele and De Alteriis, Giuseppe and Cazzaniga, Alberto and Ansuini, Alessio},
booktitle = {ICML 2024 Workshops: HiLD},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/tosato2024icmlw-emergent/}
}