• enchant spell-checker (https://abiword.github.io/enchant/)
• pip install -r requirements.txt

Optional: wandb A note about dictionaries: pyenchant works with different providers: be sure to check the requirements to install additional dictionaries. For example, for enchant, if the USER_CONFIG_DIR/PROVIDER_NAME/ is set, you can add Hunspell dictionaries (.dic and .aff files) to the directory and they will be automatically loaded. Refer to this thread for more info: pyenchant/pyenchant#167

1. Create a data directory and a model directory:

   • data directory: Where the datasets will be saved.
   • model directory: Folder of the various models.

2. Make a copy of env.example and save it as env. In env, set the value of DATA_DIR as data directory and set the value of MODEL_BASE_DIR as model directory.

The first argument is the model subdirectory name, the second are the CUDA_VISIBLE_DEVICES

• Sequential finetuning:
  • ./cill_noreplay.sh seq_finetune 0
• Joint Training
  • ./cill_joint_training.sh joint_training 0
• Replay:
  • ./cill_rp10.sh rp10_training 0
• EWC:
  • ./ewc_cill.sh ewc_training 0
• AGEM:
  • ./agem_cill.sh agem_training 0
• Self-Replay
  • ./selfrep_cill_rp20.sh selfreplay_rp20 0

Note: The scripts are set to use the iwslt2017 dataset. To use the unpc dataset, change the --dataset_name flag to unpc in the scripts and the language pairs accordingly.

Below are some of the most common cmd line flags.

To use them run generic_entrypoint.sh specifying as third argument the strategy you want to use. Strategies are under src/strategies.

Example:

./generic_entrypoint.sh model_directory 0 src.strategies.ewc_cill_unified --train_epochs 10 --dataset_name unpc

will run the EWC training with specified arguments.

Check individual strategies for more options.

Models are evaluated on validation set during training. At the end of the training phase the best model is evaluated on the test set. During training BLEU score is used as evaluation metric as it is the most common metric for machine translation and cheaper than COMET.

To evaluate a model on the test set, use the eval_models.py script under src/utils folder and pass as cmdline arguments the same used to train the model. The script will evaluate the best model using BLEU and COMET.

We provide a template Dockerfile to build an image with all the dependencies needed to run the experiments. Check the Dockerfile for more info.

If find this code useful, please consider citing our work:

@misc{resta2024selfgenerated,
      title={Self-generated Replay Memories for Continual Neural Machine Translation}, 
      author={Michele Resta and Davide Bacciu},
      year={2024},
      eprint={2403.13130},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}