Generating Cat images using StyleGAN on AWS SageMaker. Developing a simple and repeatable Data Science pipeline for Generative Adversarial Network outputs

aws s3 mb s3://thesecatsdonotexist.com

aws s3 cp index.html s3://thesecatsdonotexist.com/index.html

jupyter-notebook notebooks/catgen.ipynb

Obtain the cat gen model from the original repository (or you can get it from my s3 bucket https://s3.amazonaws.com/devopstar/resources/aws-catgen/models/karras2019stylegan-cats-256x256.pkl

aws s3 mb s3://devopstar
aws s3 cp model/karras2019stylegan-cats-256x256.pkl s3://resources/aws-catgen/models/karras2019stylegan-cats-256x256.pkl

WARNING: You will be paying > $2 per hour running this notebook. Ensure you delete it or turn it off when you aren't using it.

Create the SageMaker role that we'll attach to our SageMaker instance. Unfortunately since CloudFormation options for SageMaker do not allow us to attach Git repos as options yet.

aws cloudformation create-stack \
    --stack-name "cat-gen-sagemaker-role" \
    --template-body file://cloudformation/sagemaker_role.yaml \
    --parameters ParameterKey=S3BucketName,ParameterValue="devopstar" \
    --capabilities CAPABILITY_IAM

Once the role has been created successfully, retrieve the ARN for the use in the steps to follow.

aws cloudformation describe-stacks --stack-name "cat-gen-sagemaker-role" \
    --query 'Stacks[0].Outputs[?OutputKey==`MLNotebookExecutionRole`].OutputValue' \
    --output text

It will look something like arn:aws:iam::XXXXXXXXXXXX:role/cat-gen-sagemaker-role-ExecutionRole-PZL3SA3IZPSN.

Next create a Code repository and pass it in the repo https://github.com/t04glovern/stylegan

aws sagemaker create-code-repository \
    --code-repository-name "t04glover-stylegan" \
    --git-config '{"Branch":"master", "RepositoryUrl" : "https://github.com/t04glovern/stylegan" }'

Finally create the notebook instance ensuring you pass in the Role ARN from before, and the default code repository we just created.

aws sagemaker create-notebook-instance \
    --notebook-instance-name "cat-gen" \
    --instance-type "ml.p2.xlarge" \
    --role-arn "arn:aws:iam::XXXXXXXXXXXXX:role/cat-gen-sagemaker-role-ExecutionRole-PZL3SA3IZPSN" \
    --default-code-repository "t04glover-stylegan"

Once completed, open in JupyterLab

You should be presented with the StyleGAN repository that we set as the default when creating the repository. Open the catgen notebook

Select the kernel to use, in our case its conda_tensorflow_p36

Begin to execute the notebook using the controls at the top of the notebook, you might run into some issues when bringing in the role; I've found this to be a benign issue through.

Finally after loading in the Pickle, we generate cats by running the last section over and over again.

Due to costs being really high, we need to destory the resources when we aren't using them. Start by shutting down the notebook (you can also delete it if you want)

aws sagemaker delete-notebook-instance \
    --notebook-instance-name "cat-gen"

aws sagemaker delete-code-repository \
    --code-repository-name "t04glover-stylegan"

aws cloudformation delete-stack \
    --stack-name "cat-gen-sagemaker-role"