RUN reproducibility on VPS about continual-learning-baselines HOT 5 CLOSED

I tested a first version on the VPS. The script launches all the tests and if any fails it creates a PR here with the logs.
I plan to execute the script one time per month, by using the avalanche master version (pip installed with git+https).
In this way, we always know the last working commit of avalanche (the one related to the last month in which tests succeeded).

from continual-learning-baselines.

Yes, we should also link the reproducibility results to a specific Avalanche commit or version. Also, we should make explicit how to setup the environment and install dependencies correctly (I don't see this in the readme).

from continual-learning-baselines.

We can easily do that on the VPS. In case results are not reproduced, we can open (better if automatically) an issue here with the report.
I also updated the README with basic requirements to run the experiments. We can provide a .env file for the future, but I don't see an immediate need for that.

from continual-learning-baselines.

The readme should explain this as well I guess! At the moment it seems we guarantee reproducible results only with the packaged avalanche Beta!

from continual-learning-baselines.

Yes, because it is true for the moment. I will update it as soon as the tests succeed with the latest version.

from continual-learning-baselines.