Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition

Kai Wang, Xialei Liu, Andy Bagdanov, Luis Herranz, Shangling Jui, Joost van de Weijer

CVPRW 2022 pp. 3728-3738

doi:10.1109/CVPRW56347.2022.00417 /cvprw/2022/wang2022cvprw-incremental/

Abstract

In this paper we consider the problem of incremental meta-learning in which classes are presented incrementally in discrete tasks. We propose Episodic Replay Distillation (ERD), that mixes classes from the current task with exemplars from previous tasks when sampling episodes for meta-learning. To allow the training to benefit from a large as possible variety of classes, which leads to more generalizable feature representations, we propose the cross-task meta loss. Furthermore, we propose episodic replay distillation that also exploits exemplars for improved knowledge distillation. Experiments on four datasets demonstrate that ERD surpasses the state-of-the-art. In particular, on the more challenging one-shot, long task sequence scenarios, we reduce the gap between Incremental Meta-Learning and the joint-training upper bound from 3.5% / 10.1% / 13.4% / 11.7% with the current state-of-the-art to 2.6% / 2.9% / 5.0% / 0.2% with our method on Tiered-ImageNet / Mini-ImageNet / CIFAR100 / CUB, respectively.

PDF CVPRW Semantic Scholar

Cite

Text

Wang et al. "Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022. doi:10.1109/CVPRW56347.2022.00417

Markdown

[Wang et al. "Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022.](https://mlanthology.org/cvprw/2022/wang2022cvprw-incremental/) doi:10.1109/CVPRW56347.2022.00417

BibTeX

@inproceedings{wang2022cvprw-incremental,
  title     = {{Incremental Meta-Learning via Episodic Replay Distillation for Few-Shot Image Recognition}},
  author    = {Wang, Kai and Liu, Xialei and Bagdanov, Andy and Herranz, Luis and Jui, Shangling and van de Weijer, Joost},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2022},
  pages     = {3728-3738},
  doi       = {10.1109/CVPRW56347.2022.00417},
  url       = {https://mlanthology.org/cvprw/2022/wang2022cvprw-incremental/}
}