This repository contains the PyTorch implementation for "Mnemonics Training: Multi-Class Incremental Learning without Forgetting".

• Introduction
• Dependencies
• Running Experiments
• Citation
• Acknowledgements

Multi-Class Incremental Learning (MCIL) aims to learn new concepts by incrementally updating a model trained on previous concepts. However, there is an inherent trade-off to effectively learning new concepts without catastrophic forgetting of previous ones. To alleviate this issue, it has been proposed to keep around a few examples of the previous concepts but the effectiveness of this approach heavily depends on the representativeness of these examples. This paper proposes a novel and automatic framework we call mnemonics, where we parameterize exemplars and make them optimizable in an end-to-end manner. We train the framework through bilevel optimizations, i.e., model-level and exemplar-level. We conduct extensive experiments on three MCIL benchmarks, CIFAR-100, ImageNet-Subset and ImageNet, and show that using mnemonics exemplars can surpass the state-of-the-art by a large margin. Interestingly and quite intriguingly, the mnemonics exemplars tend to be on the boundaries between classes.

Figure: The t-SNE results of three exemplar methods in two phases. The original data of 5 colored classes occur in the early phase. In each colored class, deep-color points are exemplars, and light-color ones show the original data as reference of the real data distribution. Gray crosses represent other participating classes, and each cross for one class. We have two main observations. (1) Our approach results in much clearer separation in the data, than random (where exemplars are randomly sampled in the early phase) and herding (where exemplars are nearest neighbors of the mean sample in the early phase). (2) Our learned exemplars mostly locate on the boundaries between classes.

• Python >= 3.6
• PyTorch >= 0.4.0
• Pillow >= 6.0.0
• tensorboardX
• tqdm
• scipy

For more details, please kindly refer to this link.

git clone [email protected]:yaoyao-liu/mnemonics-training.git

cd ./mnemonics-training/1_train
python main.py --method=baseline --nb_cl=10
python main.py --method=baseline --nb_cl=5
python main.py --method=baseline --nb_cl=2

cd ./mnemonics-training/1_train
python main.py --method=mnemonics --nb_cl=10
python main.py --method=mnemonics --nb_cl=5
python main.py --method=mnemonics --nb_cl=2

We find some bugs in the code for LUCIR w/ ours.
We'll update it later after we've fixed this issue.

Please cite our paper if it is helpful to your work:

@inproceedings{liu2020mnemonics,
author    = {Liu, Yaoyao and Su, Yuting and Liu, An{-}An and Schiele, Bernt and Sun, Qianru},
title     = {Mnemonics Training: Multi-Class Incremental Learning without Forgetting},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
pages     = {12245--12254},
year      = {2020}
}

Our implementation uses the source code from the following repositories:

• Learning a Unified Classifier Incrementally via Rebalancing

• iCaRL: Incremental Classifier and Representation Learning

• Dataset Distillation

• Generative Teaching Networks