Learning Equi-Angular Representations for Online Continual Learning
Abstract
Online continual learning suffers from an underfitted solution due to insufficient training for prompt model updates (e.g. single-epoch training). To address the challenge we propose an efficient online continual learning method using the neural collapse phenomenon. In particular we induce neural collapse to form a simplex equiangular tight frame (ETF) structure in the representation space so that the continuously learned model with a single epoch can better fit to the streamed data by proposing preparatory data training and residual correction in the representation space. With an extensive set of empirical validations using CIFAR-10/100 TinyImageNet ImageNet-200 and ImageNet-1K we show that our proposed method outperforms state-of-the-art methods by a noticeable margin in various online continual learning scenarios such as disjoint and Gaussian scheduled continuous (i.e. boundary-free) data setups.
Cite
Text
Seo et al. "Learning Equi-Angular Representations for Online Continual Learning." Conference on Computer Vision and Pattern Recognition, 2024. doi:10.1109/CVPR52733.2024.02259Markdown
[Seo et al. "Learning Equi-Angular Representations for Online Continual Learning." Conference on Computer Vision and Pattern Recognition, 2024.](https://mlanthology.org/cvpr/2024/seo2024cvpr-learning/) doi:10.1109/CVPR52733.2024.02259BibTeX
@inproceedings{seo2024cvpr-learning,
title = {{Learning Equi-Angular Representations for Online Continual Learning}},
author = {Seo, Minhyuk and Koh, Hyunseo and Jeung, Wonje and Lee, Minjae and Kim, San and Lee, Hankook and Cho, Sungjun and Choi, Sungik and Kim, Hyunwoo and Choi, Jonghyun},
booktitle = {Conference on Computer Vision and Pattern Recognition},
year = {2024},
pages = {23933-23942},
doi = {10.1109/CVPR52733.2024.02259},
url = {https://mlanthology.org/cvpr/2024/seo2024cvpr-learning/}
}