conference creation fails under load due to port contention about jitsi-videobridge HOT 10 CLOSED

Not sure about a proper long-term solution, but a workaround is to use the single-port mode, in which case the portBase value will be (effectively) not used.

from jitsi-videobridge.

Thanks @bgrozev, we'll check out this mode.

from jitsi-videobridge.

@bgrozev the last time I looked the single port harvester just added this additional port. Is there a way to disable anything but that port?

from jitsi-videobridge.

@fippo that's pretty much the default behavior actually. The harvester itself only adds a port, but there is logic in the bridge which disables the dynamically allocated candidates if single-port is in use. This way we only use the single port for browsers, but use dynamic ports for endpoints without rtcpmux support (i.e. jigasi).

from jitsi-videobridge.

@bgrozev hah, that looks much better than what I remembered. Thanks!

from jitsi-videobridge.

d6610fd provides a fix for a related issue (which may have actually cause what you observed). The TCP and/or "single-port" port were used when updating the value, which resulted in it always staying at "minPort". The race condition is still there, but I believe it is very unlikely to cause any problems in practice.

from jitsi-videobridge.

Closing because no further problems have been reported for over a year. Please reopen if necessary.

from jitsi-videobridge.

We moved to single port mode, which has worked mostly fine for us and doesn't have this port-search issue.

We did have to increase the socket OS receive buffer sizes for the single port harvester. With the default limits, we were saturating these buffers and this led to audio drop outs. The settings we now use are as follows:
sysctl -w net.core.rmem_default=20971520
sysctl -w net.core.rmem_max=33554432
sysctl -w net.core.wmem_default=65536
sysctl -w net.core.wmem_max=33554432

from jitsi-videobridge.

from jitsi-videobridge.

Thanks, thats a good tip. Perhaps it could be added to the official documentation, here: https://github.com/jitsi/jitsi-videobridge/blob/master/doc/single-port.md
I'm not sure what the recommended value would be. We chose 20MB based on empirical tests with bots and how much memory our systems had available.

We've also experimented with a custom modification where we have multiple single port harvesters (i.e one per core), on different ports. The idea being that each SPH can use a smaller buffer size and gets its own IO input thread. However, we've never observed any benefit from that, except for perhaps a small improvement in ice candidate selection (less TCP and more direct and TURN-UDP in an videobridge under load.)

from jitsi-videobridge.