Hey,

Is it possible to use std::vector as a type of the queue?
I guess not, since the queue is allocating memory depending on the type and it probably can't know how big the vectors will be.

In case you wonder, I'm trying to solve the following problem with your queue:

Thread A writes data in chunks (this would be my vector) and once it's done with a chunk
Thread B is allowed to read a chunk

The problem is that Thread A's chunks are of variable size and Thread B shall only read a chunk once Thread A is done with a whole chunk.

That's why I'd like to insert std::vector into the queue.

Hi,
The code that generated this error:

auto asyncMonitor =[](moodycamel::BlockingReaderWriterQueue<std::string> &q) {
    while (true) {
        std::this_thread::sleep_for(std::chrono::milliseconds(1000));
        std::cout<<std::chrono::system_clock::now().time_since_epoch().count() << std::endl;
        if(q.size_approx()<MAX_ITEMS/10) {
            std::cout<<"adding elements ...";
            for(int i=0;i<MAX_ITEMS/2;i++) {
                bool succeeded = q.try_enqueue(sole::uuid4().str());
                //std::cout<<"adding UUID element:"<<i<<std::endl;
            }
        }
    }
};

The code is invoked as follows:

auto dummy2=std::async(std::launch::async,[&q](){
    asyncMonitor(q);
});

Any idea why this is happening?

I conducted some further tests and found an interesting tidbit in my C++ code. I used two pthreads (one reader and one writer) to transfer some data with the lock free queue object declared global. When I run both pthreads and there is NO millisleep function in writer thread between each write (or reader thread), everything works fine! I am able to read data correctly in the reader thread. Then I introduce a millisleep 100 ms in writer thread between each writes and the reader is NOT able to read the data anymore. Any idea why? Have you conducted such a test? I would think declaring global queue should not be a problem (it is extern but should not be a problem).

Run the following code by changing DELAY macro to 0 or 1. No delay behaviour seems odd. Disqus screwed my indentation so use some program like 'astyle' to get it back.

//===================== code starts below ========================

//Quick and dirty test by virgoptrex for Cameron's MoodyCamel Lock Free Queue
//compile using: g++ -g pthread_test.cc -o pthread_test -lpthread -std=c++0x

//Change DELAY value to 1 and 0 to toggle between delay and no delay

define __STDC_FORMAT_MACROS

include <inttypes.h>

include <pthread.h>

include <stdio.h>

include <stdlib.h>

include <stdint.h>

include "readerwriterqueue.h"

ifndef DELAY

define DELAY 1

endif

moodycamel::ReaderWriterQueue<uint64_t> queue;

void milliseconds_delay(uint32_t milisec)
{
struct timespec req = {0};
req.tv_sec = 0;
req.tv_nsec = milisec * 1000000L;
nanosleep(&req, (struct timespec *)NULL);
}

void *Writer(void *threadid)
{
long tid;
uint64_t counter = 0;
tid = (long)threadid;
printf("Hello World! It's me, thread #%ld!\n", tid);
while(1) {
queue.try_enqueue(counter++);
printf("sent uint64_t : %" PRIu64 "\n", counter);

if DELAY

    milliseconds_delay(100);

endif

}
pthread_exit(NULL);

}

void *Reader(void *threadid)
{
long tid;
tid = (long)threadid;
printf("Hello World! It's me, thread #%ld!\n", tid);
while(1) {
uint64_t value;
queue.try_dequeue(value);
printf("received uint64_t : %" PRIu64 "\n", value);

if DELAY

    milliseconds_delay(100);

endif

}
pthread_exit(NULL);

}

int main (int argc, char *argv[])
{
pthread_t reader_thread;
pthread_t writer_thread;
int rc;
long t;
printf("In main: creating reader thread %ld\n", t);
rc = pthread_create(&reader_thread, NULL, Reader, (void *)0);
if (rc) {
printf("ERROR; return code from reader pthread_create() is %d\n", rc);
exit(-1);
}
printf("In main: creating thread %ld\n", t);
rc = pthread_create(&writer_thread, NULL, Writer, (void *)1);
if (rc) {
printf("ERROR; return code from writer pthread_create() is %d\n", rc);
exit(-1);
}
// Last thing that main() should do
pthread_exit(NULL);
}

This might be a odd question, but what is the best way to create a vector of the ReaderWriterQueue?
I want to do this because I have a single reader, and multiple writers, but I want to keep the data from each writer separate. I've tried

vector<ReaderWriterQueue<FrameTimestamp> > matQueue;
matQueue.push_back(ReaderWriterQueue<FrameTimestamp> tempQueue(30));
matQueue.push_back(tempQueue);

but that throws error: ‘moodycamel::ReaderWriterQueue<T, MAX_BLOCK_SIZE>::ReaderWriterQueue(const moodycamel::ReaderWriterQueue<T, MAX_BLOCK_SIZE>&) [with T = frameTimestamp_t; long unsigned int MAX_BLOCK_SIZE = 512ul]’ is private

Hi,
I have an infinite stream of messages which I enqueue using a producer.
Now, I create a consumer in a separate thread like so:
``
auto qSubscriber=[](moodycamel::ReaderWriterQueuestd::string &q, zmq::socket_t *pub) {
std::cout <<"qSubscriber"<< std::endl;
std::string m;
bool succeeded;
while (true) { // How can I avoid this?
//sleep(1);
succeeded=q.try_dequeue(m);
if(succeeded) {
zmq::message_t message(m.length());
memcpy(message.data(), m.c_str(), m.length());
//std::cout << "receiving :" << m << std::endl;
pub->send(message);
}
}

};
``
Is there a way to avoid the loop and use a notification whenever there are new items in the queue?

Thanks,

Hi,

Could you please tag a release of this repository so that I can submit it as a package to https://github.com/Microsoft/vcpkg ? 👍

Hi there!

I'm looking at using this queue for some code, and it's not building under MSVS2013 x64. The issue is that the fixed-width integer types in atomicops.hpp are not available because <cstdint> is not included, apparently.

Adding #include <cstdint> to the top of atomicops.hpp fixes the issue easily.

Error   1   error C2039: 'uint64_t' : is not a member of 'std'  c:\code\xxx\src\lib\atomicops.hpp   403 1   xxx
Error   2   error C2039: 'int64_t' : is not a member of 'std'   c:\code\xxx\src\lib\atomicops.hpp   569 1   xxx
Error   3   error C2039: 'int64_t' : is not a member of 'std'   c:\code\xxx\src\lib\atomicops.hpp   634 1   xxx
Error   4   error C2039: 'uint64_t' : is not a member of 'std'  c:\code\xxx\src\lib\atomicops.hpp   403 1   xxx
Error   5   error C2039: 'int64_t' : is not a member of 'std'   c:\code\xxx\src\lib\atomicops.hpp   569 1   xxx
Error   6   error C2039: 'int64_t' : is not a member of 'std'   c:\code\xxx\src\lib\atomicops.hpp   634 1   xxx

Note that I've renamed atomicops.h atomicops.hpp to match my project's naming standards, but I can't imagine that change would cause this behaviour.

Is there a bound on how bad the approximation will be?

What I really want to do is operate with two different max sizes for the queue under two different conditions (running in background on Android needs a bigger queue, but a big queue adds a lot of latency when in foreground). I don't really want to switch to a different queue because it will cause a break in the audio. Is there way I can achieve this with size_approx(), or is calling that frequently going to severely impact performance?

Also, I'd love to run the tests on ARM, is there a single call I can do to do this, I haven't really looked through the files yet.

Hi,

I have run unittest and benchmark on ARM platform. The unittest all passed, but benchmark crashed.

May I ask you for help?

Please kindly refer to the following logs:

nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ git pull
Already up-to-date.
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git checkout -- ..." to discard changes in working directory)

    modified:   makefile

Untracked files:
(use "git add ..." to include in what will be committed)

    core

no changes added to commit (use "git add" and/or "git commit -a")
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ git diff
diff --git a/benchmarks/makefile b/benchmarks/makefile
index 5728b29..edc5126 100644
--- a/benchmarks/makefile
+++ b/benchmarks/makefile
@@ -11,7 +11,8 @@ endif
default: benchmarks$(EXT)

benchmarks$(EXT): bench.cpp ../readerwriterqueue.h ../atomicops.h ext/1024cores/spscqueue.h ext/folly/ProducerConsumerQueue.h ../tests/common/simplethread.h ../tests/common/simplethread.cpp systemtime.h systemtime.cpp makefile

•   g++ -std=c++11 -Wpedantic -Wall -DNDEBUG -O3 -g $(PLATFORM_OPTS) bench.cpp ../tests/common/simplethread.cpp systemtime.cpp -o benchmarks$(EXT) -pthread -Wl,--no-as-needed
  

•   g++ -std=c++11 -Wpedantic -Wall -DNDEBUG -g $(PLATFORM_OPTS) bench.cpp ../tests/common/simplethread.cpp systemtime.cpp -o benchmarks$(EXT) -pthread -Wl,--no-as-needed
  

run: benchmarks$(EXT)
./benchmarks$(EXT)
+
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ gdb benchmarks -c core
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/.
Find the GDB manual and other documentation resources online at:
http://www.gnu.org/software/gdb/documentation/.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from benchmarks...done.

warning: exec file is newer than core file.
[New LWP 18070]
[New LWP 17552]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
Core was generated by `./benchmarks'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x0000000000406c60 in spsc_queue::try_dequeue (this=0x7ff09094d8, v=@0x7ff0909420: 0) at ext/1024cores/spscqueue.h:82
82 v = tail_->next_->value_;
[Current thread is 1 (Thread 0x7f924df1e0 (LWP 18070))]
(gdb) bt
#0 0x0000000000406c60 in spsc_queue::try_dequeue (this=0x7ff09094d8, v=@0x7ff0909420: 0) at ext/1024cores/spscqueue.h:82
#1 0x000000000040403c in double runBenchmark<spsc_queue >(BenchmarkType, unsigned int, double&)::{lambda()#7}::operator()() const (__closure=0x22bb740)
at bench.cpp:348
#2 0x000000000040a460 in details::ArgWrapper<void, void, void>::callCallback<double runBenchmark<spsc_queue >(BenchmarkType, unsigned int, double&)::{lambda()#7}>(double runBenchmark<spsc_queue >(BenchmarkType, unsigned int, double&)::{lambda()#7}&&) const (this=0x22bb750,
callback=<unknown type in /media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks/benchmarks, CU 0x0, DIE 0x135dd>)
at ../tests/common/simplethread.h:49
#3 0x0000000000409510 in SimpleThread::CallbackWrapper<double runBenchmark<spsc_queue >(BenchmarkType, unsigned int, double&)::{lambda()#7}, details::ArgWrapper<void, void, void> >::callAndDelete(void*) (wrapper=0x22bb740) at ../tests/common/simplethread.h:71
#4 0x000000000040badc in SimpleThread::ThreadRef::threadProc (threadRef=0x22bb6c0) at ../tests/common/simplethread.cpp:50
#5 0x000000000040d30c in std::_Bind_simple<void ((SimpleThread::ThreadRef))(SimpleThread::ThreadRef*)>::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=0x22bb708)
at /usr/include/c++/5/functional:1531
#6 0x000000000040d1f8 in std::_Bind_simple<void ((SimpleThread::ThreadRef))(SimpleThread::ThreadRef*)>::operator()() (this=0x22bb708)
at /usr/include/c++/5/functional:1520
#7 0x000000000040d180 in std::thread::_Impl<std::_Bind_simple<void ((SimpleThread::ThreadRef))(SimpleThread::ThreadRef*)> >::_M_run() (this=0x22bb6f0)
at /usr/include/c++/5/thread:115
#8 0x0000007f927d9280 in ?? () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#9 0x0000007f9262dfc4 in start_thread (arg=0x7f927d9260) at pthread_create.c:335
#10 0x0000007f925a6290 in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:89
(gdb) quit
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ cat /proc/cpuinfo
processor : 0
model name : ARMv8 Processor rev 3 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 3

processor : 3
model name : ARMv8 Processor rev 3 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 3

processor : 4
model name : ARMv8 Processor rev 3 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 3

processor : 5
model name : ARMv8 Processor rev 3 (v8l)
BogoMIPS : 62.50
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x1
CPU part : 0xd07
CPU revision : 3

nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/5/lto-wrapper
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu116.04.4' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu116.04.4)
nvidia@tegra-ubuntu:/media/nvidia/DPan_SSD/ray/github/cameron314/readerwriterqueue/benchmarks$

I need to pass file paths from one thread to another and it would improve the program if from the produce thread I would add in the queue only if the item is not already there (string representing a path).

So once the consumer thread took that item from the queue it could be added again.

Hi,

Sometimes I got an assertion in "BlockingReaderWriterQueue::try_dequeue", the "assert(success);" failed !

Any idea ?

Hi,
I want to push std::shared object to lockfree queue. After some digging, I found out boost::lockfree::queue cannot use std::shared as Type. So I wander is it possible to use std::shared_ptr in this queue?

Here is my usage scenario:
Four thread, one thread produce data, other three thread each has a lock free queue.
When data production thread generate new data , I use std::shared_ptr to wrap the new data, then put
std::shared_ptr object into three consume thread's lockfree queue. Consume thread is a while loop, pop data from its lockfree queue, then read the data(no modification).

Will data pointed by std::stared_ptr correctly destructed after pop from all three lockfree queue?

It would be a nice enhancement to a speed-oriented queue in scenarios where user wants to append several items at a time. As in the answer why it is currently not happening + possible workaround.

https://stackoverflow.com/questions/1723515/insert-into-an-stl-queue-using-stdcopy

inner_enqueue(element);
should be:
inner_enqueue(std::forward(element));

try_enqueue(T&&) is the same.

Hello,
I am using this queue with one consumer and one producer thread which works great. Thank you.
Is it safe to access size_approx() of the queue from different thread than these consumer/producer threads?

I'm trying to build readerwriterqueue for Windows-x64, using VS2013. I get the following error when building the code:

error C4235: nonstandard extension used: '__asm' keyword not supported on this architecture

I did a bit of googling around and it seems that Visual Studio doesn't support inline assembly since VS2005, most people on MSDN forums say that one should use Microsoft's compiler intrinsics instead. This seems like a very un-fun rewrite, is there a way to get the code building in x64 Windows some other way?

Hi,
In vs2013 __cplusplus value is 199711. c++11 version of wait_dequeue_timed's implemention is:

#if __cplusplus > 199711L // Attempts to dequeue an element; if the queue is empty, // waits until an element is available up to the specified timeout, // then dequeues it and returns true, or returns false if the timeout // expires before an element can be dequeued. // Using a negative timeout indicates an indefinite timeout, // and is thus functionally equivalent to calling wait_dequeue. template<typename U, typename Rep, typename Period> inline bool wait_dequeue_timed(U& result, std::chrono::duration<Rep, Period> const& timeout) { return wait_dequeue_timed(result, std::chrono::duration_cast<std::chrono::microseconds>(timeout).count()); } #endif

So vs2013 program unable to use wait_dequeue_timed above, but vs2013 did support chrono. Maybe another macro can be used to check compiler's c++11 support?

by the way , bool wait_dequeue_timed(U& result, std::int64_t timeout_usecs) timeout's time unit is microsecond, right?

Expose a method which calculates the number of items that can be pushed on an empty queue without allocation.

This is useful for tuning purposes -- e.g. upon destruction one can compare initial to final queue capacities.

Excellent code. Would love to use it. There will be a problem with the license if I did. The copyright statement is fine, but the "All Rights Reserved" statement abrogates the remaining BSD license clauses. Please consider modifying by removing those three words, then the BSD license is in effect without the confusion. (i.e. either ALL rights are reserved, or the rights are spelled out by the BSD license).
Thanks.

Hello! First of all thanks for this really nice queue. I've already used it in several projects. I have a minor problem though...for example, to dequeue from the queue, let's write something like this:

MyType current;
while(queue.try_dequeue(current)) {
    // do something with current
}

But that first line doesn't work if MyType doesn't have a default constructor. So let's try it this way:

MyType* current;
while(current = queue.peek()) {
    // do something with current
    // BUT how do we actually remove it from the queue, so this isn't an endless loop?
}

The problem here is that there's no "remove" function that just removes the first element. Would it be possible to implement that? :-)

Hi @cameron314 ,

thank you for your work here! 👍 I'm wondering what is the status of the repository? as most issues got stuck 1-2 years ago.

Hi. First thanks for this wonderful lib. Using it extensively.Now, I added clear() method in my case as I occasionally empty all the data from the queue on restart of different logic. Currently I do it like this:

 inline void clear()
{
	 bool isFull = true;
	 while (isFull)
	 {
		 isFull = this->pop();
	 }
}

I wonder if you can consider adding something like this (including thread-safety/ or not) so that it becomes a part of the API?
Another moment,you allow preallocating the storage for the queue,while calling the default constructor. That's a problem if the queue is non pointer (Static) member of a class. In this case I have to switch to dynamic allocation to pass default size into the contructor? That's how I got it,unless I am missing something. Something like reserve() would be helpful
Thanks.

Hope to get around to doing this at some point - but it would be helpful to have a function wait_enqueue which would block until the queue has sufficient capacity to insert an element. This might actually be quite simple, but thought i'd post here to see if anybody else would find this useful.

It is better than adding an external condition variable (in the caller to the library) because AFAIK we can use the simple semaphore to do the same thing wait_dequeue does, except by the enqueuing thread.

Hi there,

I have modified the code to return the location of where the data is constructed/inserted in order to fill the entry data later (still by the producer... the use case is to pass the location to async ASIO code to have it filled later, to avoid using a temporary buffer and a memcpy).

I don't think it could be a problem... but I haven't thought too much about it, so please comment before I do a merge request.

Code that would be expecting bool would still compile if the change is applied, as nullptr is returned in cases false was.

Hi,

I am using your readerwriterqueue for my Android project and I compile it using g++ 4.8, which is included in the Android NDK r9c. When I compile I get alot of unused variable warnings, like this:

In file included from jni/machine.hh:53:0,
from jni/machine.cc:41:
jni/readerwriterqueue/readerwriterqueue.h: In instantiation of 'bool moodycamel::ReaderWriterQueue::try_dequeue(T&) [with T = MachineOperation]':
jni/machine.cc:952:65: required from here
jni/readerwriterqueue/readerwriterqueue.h:194:11: warning: unused variable 'nextBlockTail' [-Wunused-variable]
size_t nextBlockTail = nextBlock->tail;

Have you seen that?

       /Anton

When compiling the latest version with gcc version 7.3.0 on Ubuntu 18.04 I get the following error message:

In file included from /opt/readerwriterqueue/readerwriterqueue.h:7:0,
                 from my_source_code;
/opt/readerwriterqueue/atomicops.h:46:44: error: missing binary operator before token "("
 #if defined(__has_feature) && __has_feature(thread_sanitizer)

Reading Clang ThreadSanitizer Documentation I came up with the following fix (I'm not saying it's the best way): change the code at atomicops.h:46 from this

// AE_NO_TSAN
#if defined(__has_feature) && __has_feature(thread_sanitizer)
#define AE_NO_TSAN __attribute__((no_sanitize(thread)))
#else
#define AE_NO_TSAN
#endif

to

#define AE_NO_TSAN
#if defined(__hasi_feature) 
 #if __has_feature(thread_sanitizer)
  #undef AE_NO_TSAN
  #define AE_NO_TSAN __attribute__((no_sanitize(thread)))
 #endif
#endif

My g++ version:

$ g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 7.3.0-27ubuntu1~18.04' --with-bugurl=file:///usr/share/doc/gcc-7/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-7 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 7.3.0 (Ubuntu 7.3.0-27ubuntu1~18.04)

I just got a crash using the readerwriterqueue. This is my code:

std::string bytes;
queueHasElements = queue.try_dequeue(bytes);
if (queueHasElements) {
    // Do something with bytes

I get the following error:

Assertion failed: (!inSection), function ReentrantGuard, file some path /lib/readerwriterqueue/readerwriterqueue.h, line 546.

It's the first time this has happened, after using the queue for a long time. I would love to add the version I'm using. Where is it written in the files?

This is the crash log:

Thread 19 Crashed:
0 libsystem_kernel.dylib 0x00007fff8ef43286 __pthread_kill + 10
1 libsystem_c.dylib 0x00007fff8d440b53 abort + 129
2 libsystem_c.dylib 0x00007fff8d408c39 __assert_rtn + 321
3 com.SomeProduct.Id 0x00000001085cea2b moodycamel::ReaderWriterQueue<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, 512ul>::ReentrantGuard::ReentrantGuard(bool&) + 91 (readerwriterqueue.h:546)
4 com.SomeProduct.Id 0x00000001085ce32d moodycamel::ReaderWriterQueue<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, 512ul>::ReentrantGuard::ReentrantGuard(bool&) + 29 (readerwriterqueue.h:552)
5 com.SomeProduct.Id 0x00000001085c7cce bool moodycamel::ReaderWriterQueue<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >, 512ul>::try_dequeue<std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > >(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >&) + 78 (readerwriterqueue.h:209)
(rest of the call stack cut out)

Thread 19 crashed with X86 Thread State (64-bit):
rax: 0x0000000000000000 rbx: 0x0000000000000006 rcx: 0x000000011054bcb8 rdx: 0x0000000000000000
rdi: 0x000000000000f733 rsi: 0x0000000000000006 rbp: 0x000000011054bce0 rsp: 0x000000011054bcb8
r8: 0x0000000000000000 r9: 0x0000000000000222 r10: 0x0000000008000000 r11: 0x0000000000000206
r12: 0x000000010868c21b r13: 0x0000000108f49000 r14: 0x000000011054d000 r15: 0x000000010868c0e9
rip: 0x00007fff8ef43286 rfl: 0x0000000000000206 cr2: 0x00007fff7975bfd8

Logical CPU: 0
Error Code: 0x02000148
Trap Number: 133

When adding readerwriterqueue to my project, I get the following error in gcc:

/usr/bin/g++  -c  "/home/user/project/src/readerwriterqueue-impl-myClass.cpp" -pedantic-errors -O3 -std=c++11 -Wall -o ./Release/src_readerwriterqueue-impl-myClass.cpp.o -I. 
In file included from /home/user/project/src/readerwriterqueue.h:7:0,
                 from /home/user/project/src/readerwriterqueue-impl-myClass.cpp:1:
/home/user/project/src/atomicops.h:51:24: error: explicit instantiation shall not use 'inline' specifier [-fpermissive]
 #define AE_FORCEINLINE inline
                        ^
/home/user/project/src/readerwriterqueue-impl-myClass.cpp:8:10: note: in expansion of macro 'AE_FORCEINLINE'
 template AE_FORCEINLINE bool ReaderWriterQueue<myClass *, 512>::enqueue(myClass * const& element);
          ^

Using -fpermissive solves it but that isn't a good solution in my opinion.

Here is the gcc version information:

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.4' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs
--enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared
--enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib
--enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes
--with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin
--with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre
--enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64
--with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch
--disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic
--enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4)

I notice that atomicops.h implements atomic_*_fence on all _MSC_VER

Is there any benefit to make this conditional on say _MSC_VER < 1900 as I think its all present and correct in <atomic> for VS2015

The unit tests / stab tests segfault for when compiled with -static on gcc 4.7.2 (Debian Wheezy) and gcc 4.9.2 (Debian Jessie). This is apparently due to a failure to link properly with pthreads when statically linked. Compiling with -Wl,-u,pthread_cancel,-u,pthread_cond_broadcast,-u,pthread_cond_destroy,-u,pthread_cond_signal,-u,pthread_cond_wait,-u,pthread_create,-u,pthread_detach,-u,pthread_cond_signal,-u,pthread_equal,-u,pthread_join,-u,pthread_mutex_lock,-u,pthread_mutex_unlock,-u,pthread_once,-u,pthread_setcancelstate resolves the segfault by forcing linking, it seems. I make this change in cecd5c0.

Alternatively, simply don't use -static.

As Jessie was only just released, people may be running into this for a few years, so hopefully this will save someone some grief.

below are the performance execution time for the call try_dequeue () (time in microsecond for performance measurement)

0.8
0.9
0.8
0.9
1.0
0.9
5.7
5.6
12.0
6.0
7.6
5.4
1.0
1.0
1.0
0.9
0.9
0.5
1.7
2.5
8.48
11.63
0.9
0.7
0.6
0.5
9.9
0.9
1.4
0.9

Above stats shows that in a cyclic faction the execution time varies. Also, the variation is too much i.e. approx 10 times. what might could be the cause ? I am not able to figure it out.

When dequeuing on a potentially blocking queue it is often useful to implement a timeout or more generally have a means to wake up the waiting receiver ideally without sending a 'fake' packet through.

For the send side, could we expose a signal() function to call sema.signal() and remove the check in the dequeue functions that expect a woken thread to always succeed in inner.try_dequeue

This does however change the stated description of try_dequeue and wait_dequeue so I wonder if there is a better way....

[Could implement a 'dummy msg', 'timeout' and/or 'done' command packet but that affects the generic nature of the queue's payload template as we'd need to define and recognise those commands.]

AE_FORCEINLINE void signal()
{
    sema.signal();
}

template<typename U>
bool try_dequeue(U& result)
{
    if (sema.tryWait()) {
        return inner.try_dequeue(result);
    }
    return false;
}

template<typename U>
void wait_dequeue(U& result)
{
    sema.wait();
    bool success = inner.try_dequeue(result);
    // TODO what to do with result??? Can't convey 'failed' wait
}

At the moment the queue can only increase its memory consumption. Would it be possible to release allocated memory when the size() decreases or when clear() is called?

Is there an efficient way of implementing a blocking dequeue? Am I thinking about it wrong? I don't want my consumer thread spinning if there's no data.

On ubuntu 1404 LTS with gcc 4.8.2

Clone repo
cd benchmarks
make
< compiles with
g++ -std=c++11 -Wpedantic -Wall -DNDEBUG -O3 -g -lrt bench.cpp ../tests/common/simplethread.cpp systemtime.cpp -o benchmarks -pthread

./benchmarks
terminate called after throwing an instance of 'std::system_error'
what(): Enable multithreading to use std::thread: Operation not permitted

ldd benchmarks shows no pthread lib

Looks like this might be related to this bug (doesn't link pthread)
https://bugs.launchpad.net/ubuntu/+source/gcc-defaults/+bug/1228201

so added '-Wl,--no-as-needed' to makefile to work around. Now runs.

See also http://stackoverflow.com/questions/19463602/compiling-multithread-code-with-g
[Shows a simple example that fails]

Not sure if there is a better solution?
(separating compilation from linking and passing -pthread to both also works)

Looks like ubuntu/debian toolchain pass --as-needed to the linker, but the benchmark libs don't correctly propagate the need for libpthread so it gets removed at link unless we tell the linker NOT to cull 'unnecessary' libs.
http://gcc.gnu.org/ml/gcc-help/2010-12/msg00338.html

Arguably the fix is to make sure gcc doesn't assume that pthread isn't needed, rather than the workaround but I'm a bit bemused that it fails in the first place (see SO simple example that also fails)

I had to change the weak atomic to do a load() in the copy and move operators, because on my platform (linux,g++ 7,c++17) std::atomic may not be copied or moved directly.
like:
weak_atomic(weak_atomic const& other) : value(other.value.load()) { } weak_atomic(weak_atomic&& other) { value = other.value.load(); }
otherwise BlockingReaderWriterQueue would not compile while trying to move it about.
(i also deleted the copy constructor/operator on the queue instead of having it be private, but i am too lazy to undo the change and check if this also had to do with it not compiling)

I'm developing an audio application, where the (realtime) Audio thread and the GUI thread communicate via a ReaderWriterQueue. I have a queue containing objects of type PitchCurveUpdateMessage, which has a field of type std::vector<std::pair<int, ClientsidePitchData>>.
I populate the vector in the GUI thread, wrap it in a PitchCurveUpdate object and send it over to the Audio thread via queue.try_enqueue.

On the audio thread, I have a function called fetch(), which calls try_dequeue, providing a pre-allocated PitchCurveUpdateMessage as the target object to avoid memory allocation. To me, this seems like basic usage of the ReaderWriterQueue, however, XCode 10's ThreadSanitizer reports a data race in my code (see below).

Is there anything I'm missing when working with std::vectors? Do I need to protect them in any special way? Are vectors not suitable for use in a ReaderWriterQueue, as they allocate their own heap memory?

Thanks in advance for any help!

atomicops.h:477:30: error:
use of undeclared identifier 'errno'
while (rc == -1 && errno == EINTR);
and also the compiler complained undeclared identifier 'EINTR'

My build environment is Ubuntu 14.04, Clang++ 3.7.1
Ubuntu clang version 3.7.1-svn253742-1~exp1 (branches/release_37) (based on LLVM 3.7.1)
Target: x86_64-pc-linux-gnu
Thread model: posix

I tried adding #include <errno.h> in atomicops.h, still same problem.

I have an operation which involves a single producer thread, and guaranteed only a single consumer thread accessing the queue concurrently. The rub is that the consumer could theoretically change pid numbers, ie be a different thread. Does this still count as SPSC or do I have to use the multiple thread version.

Is it possible to use ReaderWriterQueue on non-primitive types (like strings), e.g.

moodycamel::ReaderWriterQueue<std::wstring> g_queue;

? This compiles in VS2010 but I am not sure if enqueue/dequeue are really thread-safe.

The reason why I am asking is that non-primitive types are possible in std::queue (but not thread-safe) and are not allowed in boost::lockfree::queue.

try_enqueue() (and enqueue()) are declared with the concrete type T. This means if you pass something convertible-to-T the conversion takes place immediately, to construct item. This is a problem when you want to std::move() the argument, as the move takes place immediately. Then if the try_enqueue() fails because queue is full, your argument has been moved-from, and the moved-to 'item' param is then destroyed.

Even if you aren't using move, this is somewhat inefficient as you incur the construction cost regardless of whether the item is enqueued or not.

Consider the concrete example of a boost::variant<std::unique_ptr, std::unique_ptr>. If you call try_enqueue(std::move(p)) and there's no space, p is now nullptr, which is perhaps unexpected.

You can work around this, but it's not pretty, e.g. boost::variant<std::unique_ptr, std::unique_ptr> v { std::move(p) }; q.try_enqueue(std::move(v)); (incidentally declaring a value_type is a good idea for containers also).

A better solution IMHO would be to provide try_emplace(), ie construct-in-place. With this the construction of the object is deferred until the new, and hence skipped entirely if the queue is full. This solves the above problem and may also be more efficient, if constructing a T is non-trivial.

I have prepared a patch for this, I'll do a PR shortly for your consideration.

Hi,
Whenever I try to do a try_dequeue on an empty readerwriter queue my code crashes at moodycamel/readerwriterqueue.h:239. Though the queue is empty it still tries to go ahead and dequeue the element. Can you please help? My code is actually looping on the try_dequeue and most of the time it works fine and suddenly it crashes at result = std::move(*element); on line 239 even though the queue is empty. Please help.

Thanks,
Siril.

HI.I am trying to understand how to use your queue with pointers to memory .You got a template param which is type.So how does one go with it when the data is a pointer to block of memory which I need to push into queue?

Where is ReentrantGuard defined?

It would be nice if readerwriterqueue had a test suite, that way external committers could verify that the core behavior is unaltered before sending pull requests. Also I find that having a test suite is a nice demonstration of a library's example usage.

Hi!
First of all, great work! I'm trying to insert a readerwriterqueue into a map but the

ReaderWriterQueue(ReaderWriterQueue const&) {}

method is private. Here is what I'm trying to do:

std::unordered_map<uint32_t, ReaderWriterQueue<int>> map;
ReaderWriterQueue<int> rwq(10000);
rwq.enqueue(10);
map.insert(std::pair<uint32_t, ReaderWriterQueue<int>>(1,rwq));

Do you know how can this be solved?
Moreover, why disable copying?
Thank you!

After chasing down a mysterious rare crash in my code, the issue turned out to be that timed_wait in atomicops.h Semaphore implementation returned true without waiting iff tv_nsec==1000000000. It seems that sem_timedwait "bombs" on that value in addition to any nsec value above that. The fix seems to be replacing

if (ts.tv_nsec > nsecs_in_1_sec) {

with

if (ts.tv_nsec >= nsecs_in_1_sec) {

This is my pseudo code using asio.

asio::strand strand;
ReaderWriterQueue<int> queue;

void doSomething()
{
  strand.post([]
  {
      int* value = queue.peek();
      // Do something
  }
}

// EntryPoint
void Enqueue(int value)
{
      queue.enqueue(value);
      Process();
}


void Process()
{
  strand.post([]
  {
      int* value = queue.peek();              // <-- assert failed!!!!!
      if(value)
      {
          // Do something
          queue.pop();
      }
  });
  doSomething();
}

doSomething() and Process() may work in different threads, but they are obviously being synchronized to asio::strand.

whats wrong?

i am trying to implementing code but getting following error

void moodycamel::BlockingReaderWriterQueue<T, MAX_BLOCK_SIZE>::wait_dequeue(U&) [with U = std::shared_ptr; T = std::shared_ptr; long unsigned int MAX_BLOCK_SIZE = 512ul]: Assertion `success' failed.

this code using inside Thread(detached)