Avoid RateLimitError about brawlstats HOT 15 CLOSED

Hey, I'm currently away right now, I'll look into it more when I get back. Thanks for using brawlstats!

from brawlstats.

Hi,
so I was also looking into this and found two possible solutions:

1. asyncio.Lock() (preferred)
   Initialize an self.lock = asyncio.Lock() in the client and modify _aget_model to

async def _aget_model(self, url, model, key=None):
  async with self.lock:
    data, resp = await self._arequest(url)
    await asyncio.sleep(1./self.ratelimit)
...

Like this, one can call the line of code above

players = await asyncio.gather(*(client.get_player(player.tag) for player in club.members))

as intended.

2. Provide another method intended for sending multiple requests, e.g. (however could be easily archived as well by using method 1)

async def _aget_models(self, urls, model, key=None):
  tasks = []
  for url in urls:
    tasks.append(asyncio.ensure_future(self._aget_model(url, model=model, key=key)))
    await asyncio.sleep(1./self.ratelimit)

  results = await asyncio.gather(*tasks)
  return results

Please let me know what you think and enjoy your holidays!

from brawlstats.

I'm assuming if I implement option 1, then users who are only requesting for 1 url will get their results 1/3 seconds slower (as the user rate limit is 3 reqs/sec). However, it would be much simpler than the second option.

from brawlstats.

Possibly yes but I guess the answer anyway takes 1/3 seconds before it arrives. And I guess one could also argue that there is no need to use it asynchronously when you only request one url. However, maybe one could also make it an option to specify a ratelimit when initialising the client (and set it to 0 if one does not want it)...

from brawlstats.

the discord.py wrapper is asynchronous and it is best practice to use async libraries for a discord bot. I could make an option when initializing the client such as prevent_ratelimit=True if you want to use the asyncio.Lock()

from brawlstats.

@burneyy added in ccbca58, should be out in v2.3.6

from brawlstats.

@SharpBit Very good! What do you think, should there also be an automatic retry after X seconds when there was a rate limit error still? It seems to still occur occasionally. Like they do here: https://github.com/Rapptz/discord.py/blob/master/discord/http.py#L182

from brawlstats.

I could try but that is currently not a high priority and I have some other stuff to do in the next week or two

from brawlstats.

I'm assuming if I implement option 1, then users who are only requesting for 1 url will get their results 1/3 seconds slower (as the user rate limit is 3 reqs/sec). However, it would be much simpler than the second option.

Good practice is to send ctx.typing in these cases

from brawlstats.

@MinerChAi I don't know if he is making a discord bot or some other application

from brawlstats.

@MinerChAI I don't know if he is making a discord bot or some other application

Nope I am not making a discord bot so I also don't know what ctx.typing is I must say..

from brawlstats.

Hey, just wanted to leave a comment that if you upgraded to v2.3.6, the wrapper probably was broken. This is due to the new API server (api.brawlapi.cf/v1) having timeout issues, if you have v2.3.6, please upgrade to v2.3.7 @burneyy @MinerChAI. Sorry for the inconvenience and the API developers are working at it.

from brawlstats.

You are right, that is odd (burneyy). I am still getting occasional RateLimitErrors that are failing the tests. When I test locally, I do not get ratelimited but it seems on travis-ci that I am. I wonder if it is hitting the global ratelimit or if it's the user ratelimit.

from brawlstats.

@burneyy Update to v2.3.8 Rate limits do happen, but it is a bit less often. I am trying to work towards no rate limits at all. However, please still update as the old Base URL will be deprecated tomorrow or in the next week and v2.3.7 and below will not work unless you specify the new base url when creating the client

from brawlstats.

Was requested to be closed.

from brawlstats.