Running multiple instances about spotty HOT 3 CLOSED

Hi @maclandrol,

Thank you! :)

All spotty commands have the -c/--config parameter. So you can start several instances by providing different spotty configuration files to the spotty start command, and then run your script on each instance providing the same configuration file to the spotty run command. For example:

spotty start -c spotty.i1.yaml
spotty run train -c spotty.i1.yaml

spotty start -c spotty.i2.yaml
spotty run train -c spotty.i2.yaml

And don't forget to stop all of them after :).

I know that it's not very convenient, because you're basically copy-pasting the same configuration file several times, and then using the -c parameter for every command. Moreover, the project name parameter should be different in each configuration file, otherwise, the tool tells you that the stack already exists. Also, it will create a new S3 bucket for each configuration.

I faced this problem before as well, and I've made it slightly better it in the future version 1.2. The format of the configuration file will be a little bit different, and you will be able to specify there several instances. They will reuse the same S3 bucket. Also, scripts will be parameterizable, so you could use custom parameters: spotty run [INSTANCE_NAME] SCRIPT_NAME [-p PARAM1=VAL1 PARAM2=VAL2 ...].

from spotty.

This answers all my question. Thanks !

I am using spotty inside a wrapper for automatic config generation and instance launching, so it should be fine.

Reusing the same S3 bucket will be a really great feature. It might also be helpful to have a way to automatically stop the instance after a given time limit (I am using a scheduler now, which is working fine) !

from spotty.

Thank you for the feedback! I will add the time limit feature to my TODO list.

from spotty.