Weird issue when kicking off multiple workers about mosquito HOT 11 CLOSED

I'm sure. We just launched this in to production yesterday, so we're just now seeing it. I'm gonna do more digging in, but you know how it goes with the lack of debugging tools in crystal 😅 I'll let you know if I come up with any additional info.

from mosquito.

I did indeed, and I did a horrible job with that 😢 sorry about that. I tried a few different things, but I was never able to recreate it outside of the app. My guess is we must have been doing something weird.

You can close this out for now if you'd like, and if I end up getting a better code example, I can always just reopen it.

from mosquito.

@jwoertink In the example you gave, are you saying that it only runs the PasswordWorker, but not EmailWorker, LoginWorker, etc?

from mosquito.

In that example, I'm saying that it may run all 4, or it may run just the LoginWorker, or it may run only the PasswordWorker and the PostbackWorker.... Like every permutation of those. It doesn't really make sense, but my guess is all of them get shoved in to redis, but only some get pulled back out.

from mosquito.

@jwoertink wow, that's significant. Thanks for reporting. I have a feeling it's going to be difficult to reproduce.

from mosquito.

@jwoertink can you go into more detail about your deployment? I'm curious how many task runners you have, for example.

from mosquito.

oh, I didn't even think about that.

require "./shards"
require "./workers/*"

Mosquito::Runner.start

I'm just running

crystal build src/mosquito.cr --release -o worker
./worker

So I never actually specified the number of workers or anything. What does that config look like?

from mosquito.

By default it's just the one worker. I meant spinning up ./worker more than once, possibly on multiple boxes.

from mosquito.

We only have the one server, so only 1 process for that running.

from mosquito.

@jwoertink you mentioned you might do some digging, did anything get dug up?

from mosquito.

No worries! Thanks for checking back in. Please re-open if needed.

from mosquito.