This is the official repository of the ICML, 2023 paper "BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning" by Kishaan Jeeveswaran, Prashant Bhat, Bahram Zonooz and Elahe Arani.

TLDR: A novel representation rehearsal-based continual learning approach that, by incorporating constructive noises at various stages of the vision transformer and regularization, enables effective and memory-efficient Continual Learning.

OUTPUT_DIR: Directory to save output contents.
DATA_DIR: Directory containing the datasets.

• CIFAR-100
• ImageNet-100
• Tiny ImageNet

To train BiRT on CIFAR-100 dataset 10 tasks setting with buffer size 500:

python main.py --seed 42 --options options/data/cifar100_10-10.yaml options/data/cifar100_order1.yaml options/model/cifar_birt.yaml --data-path <DATA_DIR> --output-basedir <OUTPUT_DIR> --base-epochs 500 --batch_mixup --batch_logitnoise --ema_alpha 0.001 --ema_frequency 0.003 --distill_version l2 --distill_weight 0.05 --distill_weight_buffer 0.001 --rep_noise_weight 1.0 --repnoise_prob 0.5 --finetune_weight 2 --representation_replay --replay_from 1 --sep_memory --num_workers 8 --csv_filename results.csv --memory-size 500 --tensorboard --epochs 500

If you find the code useful in your research please consider citing our paper:

@article{jeeveswaran2023birt,
  title={BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning},
  author={Jeeveswaran, Kishaan and Bhat, Prashant and Zonooz, Bahram and Arani, Elahe},
  journal={arXiv preprint arXiv:2305.04769},
  year={2023}
}