Faktory crashes with lots of connections under OSX about faktory HOT 5 CLOSED

I did ulimit -n 1024 to increase the max open files in both terminals. No errors at ~240 threads. They go up quickly once I approach and pass 256. I wonder if Faktory's one goroutine per connection maxes out on OSX at several hundred connections with no tuning. It would be interesting to dive into Linux and see its limits and what tunings might be useful. Redis might be good to dive into to see what knobs they twiddle (although being single threaded does change things a bit for them). Just some random thoughts.

from faktory.

This happens to me on Linux too (with 200), and I have the limit set to 8192 already.

from faktory.

Can you check the actual limits of your process in /proc/$PID/limits ? they might be
different from what you see in your shell.

from faktory.

They're actually even higher than my limits (presumably the Go runtime does something funky). It could of course be that the limits are set differently inside the Docker container, and that that's where the issue arises?

from faktory.

The latest Faktory appears to work fine on macOS as long as I run ulimit -n 1024 in each terminal first. I suspect the move to Redis helped reduce the number of open files.

$ go run test/load/main.go 30000 100
Running loadtest with 30000 jobs and 100 threads
Processed 30049 pushes and 30049 pops in 6.979729 seconds, rate: 4298.161438 jobs/s
$ go run test/load/main.go 30000 200
Running loadtest with 30000 jobs and 200 threads
Processed 30099 pushes and 30099 pops in 6.728305 seconds, rate: 4458.774980 jobs/s
$ go run test/load/main.go 30000 300
Running loadtest with 30000 jobs and 300 threads
Processed 30149 pushes and 30149 pops in 6.977836 seconds, rate: 4299.327217 jobs/s
$ go run test/load/main.go 30000 400
Running loadtest with 30000 jobs and 400 threads
Processed 30199 pushes and 30199 pops in 7.094742 seconds, rate: 4228.483610 jobs/s
$ go run test/load/main.go 30000 500
Running loadtest with 30000 jobs and 500 threads
Processed 30249 pushes and 30249 pops in 7.269181 seconds, rate: 4127.012486 jobs/s

from faktory.