A Tale of Two CILs: The Connections Between Class Incremental Learning and Class Imbalanced Learning, and Beyond

Abstract

Catastrophic forgetting, the main challenge of Class Incremental Learning, is closely related to the classifier’s bias due to imbalanced data, and most researchers resort to empirical techniques to remove the bias. Such anti-bias tricks share many ideas with the field of Class Imbalanced Learning, which encourages us to reflect on why these tricks work, and how we can design more principled solutions from a different perspective. In this paper, we comprehensively analyze the connections and seek possible collaborations between these two fields, i.e. Class Incremental Learning and Class Imbalanced Learning. Specifically, we first provide a panoramic view of recent bias correction tricks from the perspective of handling class imbalance. Then, we show that an adapted post-scaling technique which originates from Class Imbalanced Learning is on par with or even outperforms SOTA Class Incremental Learning method. Visualization via violin plots and polar charts further sheds light on how SOTA methods address the class imbalance problem from a more intuitive geometric perspective. These findings may encourage further infiltration between the two closely connected fields, but also raise concerns about whether it is correct that Class Incremental Learning degenerates into a class imbalance problem.