zflamig / dask-era5 Goto Github PK

Hi! I've been trying to setup a Fargate cluster with your cloud formation.

Currently experiencing some difficulties connecting to the scheduler.

So, the dask scheduler port specified in the cloudformation is in fact 8786 and it is the same in the code block:

client = Client('Dask-Scheduler.local-dask:8786')
client

Have you experienced issues in the past @zflamig? Strung out on debugging options for the moment.

Please let me know if you have any ideas.

Thanks!

I'm guessing I missed a step or something, but my notebook failed when it hit the dask compute, I'm assuming due to mismatched versions:

https://gist.github.com/rsignell-usgs/91c986838599f4c5da5bf7ee777c29a9

Resource of type 'AWS::Logs::LogGroup' with identifier '{"/properties/LogGroupName":"/aws/codebuild/era5"}' already exists.

@zflamig , the CodeBuild pipeline failed due to dockerhub pull rate limit on miniconda

Step 1/9 : FROM continuumio/miniconda3:4.7.12
347 | toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit
348

@ocefpaf, do you know if there is a different place to pull the Miniconda Container from now?

I tried generating a GitHub access token to try deploying the stack. However, I cannot add it to the deployment without having an error:

Parameter 'DaskWorkerGitToken' must match pattern [a-zA-Z0-9]+

Is the pattern matching something custom that worked before but not anymore with the tokens? Is there some other way to generate a token matching the specifications?

Let me know if you need any more information.

Thanks!