Towards Understanding Catastrophic Forgetting in Two-Layer Convolutional Neural Networks
Abstract
Continual learning (CL) focuses on the ability of models to learn sequentially from a stream of tasks. A major challenge in CL is catastrophic forgetting (CF). CF is a phenomenon where the model experiences significant performance degradation on previously learned tasks after training on new tasks. Although CF is commonly observed in convolutional neural networks (CNNs), the theoretical understanding about CF within CNNs remains limited. To fill the gap, we present a theoretical analysis of CF in a two-layer CNN. By employing a multi-view data model, we analyze the learning dynamics of different features throughout CL and derive theoretical insights. The findings are supported by empirical results from both simulated and real-world datasets.
Cite
Text
Li et al. "Towards Understanding Catastrophic Forgetting in Two-Layer Convolutional Neural Networks." Proceedings of the 42nd International Conference on Machine Learning, 2025.Markdown
[Li et al. "Towards Understanding Catastrophic Forgetting in Two-Layer Convolutional Neural Networks." Proceedings of the 42nd International Conference on Machine Learning, 2025.](https://mlanthology.org/icml/2025/li2025icml-understanding/)BibTeX
@inproceedings{li2025icml-understanding,
title = {{Towards Understanding Catastrophic Forgetting in Two-Layer Convolutional Neural Networks}},
author = {Li, Boqi and Wang, Youjun and Liu, Weiwei},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
year = {2025},
pages = {36057-36095},
volume = {267},
url = {https://mlanthology.org/icml/2025/li2025icml-understanding/}
}