Time between requests to prevent Hetzner Rate Limit about docker-machine-driver-hetzner HOT 5 CLOSED

Hi,

first of all, thank you for bringing this up. Could you perhaps describe the process that got you into the rate limit a little bit more?

While the 429 error absolutely makes sense, the message seems a bit odd to me: It reads more like a resource exhaustion in the given region, rather than you 'spamming' the cloud API.

If it is due to an external reason, IMO the fail-fast option (i.e. the current behaviour) is the best way. On the other hand, if it comes down to requests being sent too frequently the driver could indeed slow itself down.

I'll have to look, whether the status code is actually exposed in the error interface returned by the hcloud library; if not, I don't really see a way of handling this gracefully.

If it is available, would it be ok for your use-case to make this an explicit opt-in as a command line option? The driver is being used in automated scenarios as well and I think for the non-manual case, failing fast is preferrable to waiting an unspecified time.

Further discussion welcome.

from docker-machine-driver-hetzner.

The actual error from the driver is
ERROR: Error creating machine: Error in driver during machine creation: could not create server: we are unable to provision servers for this location, try with a different location or try later (resource_unavailable) driver=hetzner

I am not sure if the rate limiting is the consequence of the driver retrying to create machines or vice versa.

from docker-machine-driver-hetzner.

I totally agree. The Hetzner Support took quite long to let me know that their cloud actually ran out of resources. Me trying to boot new instances led to the 429 errors which are connected to the account you're using, not an api key or login. So my whole team suddenly got the 429ers.

It was a combination of low resources and, afterwards, my CLI tool going wild. So I would agree if you'd say that it was an unusal situation. I'd treat it as super low priority if I would be you.

If it takes 5 minutes to implement an additional --time-between-calls parameter, then I'd be very happy. Otherwise I just keep my fingers cross that hetzner now has enough resources in all regions.

from docker-machine-driver-hetzner.

Unfortunately we had to move back to root servers since the performance of the cloud servers wasn't satisfying. Thanks for your support!

from docker-machine-driver-hetzner.

Unfortunately we had to move back to root servers since the performance of the cloud servers wasn't satisfying. Thanks for your support!

Sorry to hear that but thank you for taking the time to come back and close this. Perhaps this could still be implemented in the future, but wrapping every operation in a retry-loop (and maintaining a consistent internal state) turned out to be harder than I initially anticipated, especially with cleanup stuff. I will keep a mental note, but with your initial issue being an external problem, I don't think the benefit outweighs the additional complexity brought in.

Still glad, you found something working for you in the end, even if it turned out to be the good old bare metal.

from docker-machine-driver-hetzner.