Overcoming Catastrophic Interference Using Conceptor-Aided Backpropagation
Abstract
Catastrophic interference has been a major roadblock in the research of continual learning. Here we propose a variant of the back-propagation algorithm, "Conceptor-Aided Backprop" (CAB), in which gradients are shielded by conceptors against degradation of previously learned tasks. Conceptors have their origin in reservoir computing, where they have been previously shown to overcome catastrophic forgetting. CAB extends these results to deep feedforward networks. On the disjoint and permuted MNIST tasks, CAB outperforms two other methods for coping with catastrophic interference that have recently been proposed.
Cite
Text
He and Jaeger. "Overcoming Catastrophic Interference Using Conceptor-Aided Backpropagation." International Conference on Learning Representations, 2018.Markdown
[He and Jaeger. "Overcoming Catastrophic Interference Using Conceptor-Aided Backpropagation." International Conference on Learning Representations, 2018.](https://mlanthology.org/iclr/2018/he2018iclr-overcoming/)BibTeX
@inproceedings{he2018iclr-overcoming,
title = {{Overcoming Catastrophic Interference Using Conceptor-Aided Backpropagation}},
author = {He, Xu and Jaeger, Herbert},
booktitle = {International Conference on Learning Representations},
year = {2018},
url = {https://mlanthology.org/iclr/2018/he2018iclr-overcoming/}
}