multithreading stuck when the number of threads is too high about vvdec HOT 4 CLOSED

The issue has a workaround solution in current master (but not yet in v1.6.0). We will have look into a proper fix at some point.

from vvdec.

I am also encountering this issue using 2.0.0 release which has the workaround (max threads limit to 64). I am seeing this on a 32 core (64 thread) machine with AMD EPYC 7543 CPU

It looks like one thread is stopped waiting in

m_allThreadsWaitingCV.wait( lock, [=] { return !m_allThreadWaitingMoreWork; } );

which means the thread pool has been paused.

I don't know this code at all but a quick scan of it tells me possibly this is what's happening:

The thread pool can be paused after barrier task for the final packet in the stream has arrived, and then it will never wake up because PoolPause::unpauseIfPaused won't be called again unless there is another call to vvdec_decode with more data (which might not happen).

This can happen if the final barrier task arrives in between the call to findNextTask and m_poolPause.pauseIfAllOtherThreadsWaiting();

from vvdec.

Interesting, thanks for the analysis.

Now for my understanding. With which number of threads did you encounter the issue? So the problem happens at the end of the decoding before flushing, or after flushing? (but for sure after the last packet has been passed to the decoder?)

from vvdec.

This issue seems to be fixed with the latest reworking of the thread pool implementation.

Please reopen the issue if the problem persists.

from vvdec.