tchaloupka / during Goto Github PK

Hi, @tchaloupka!
do you plane update this library for newer kernel version (5.7 for example)?

1. There is no reason not to
2. Having user_data == 0 (as it's now), will confuse my program, because 0 is a valid userData for normal requests (I am using ulong.max as magic value).
3. Allows to handle cancels like other requests, e.g. return the result code etc.

Would you accept a PR?

Hello @tchaloupka

Can you please help? Below you'll find simple code which demonstrate my problem. This code has two threads - main thread just submits NOP entries to SQ in a tight loop and Consumer thread consume these NOPs from CQ.
Expected result - program finish with next output:

Running ./test 
completions=10000000
submissions=10000000
overflow=   0

but instead I get some number of overflows and consumer loop never ends.
If I increase RING_SIZE to 4096 then I get expected result. So the question is - how can I avoid overflows and event loss.

Thanks!

$ uname -a
Linux igor-Zen 5.3.0-53-generic #47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

import std.stdio;
import std.datetime;
import core.thread;
import during;

static long submissions, completions;
enum iterations = 10_000_000;
enum RING_SIZE = 32; 
void main()
{
    Uring io; 
    io.setup(RING_SIZE);

    void Consumer()
    {   
        // consume NOP events
        while(completions<iterations)
        {
            int r = io.wait(1);
            if(r==0)
            {
                io.popFront;
                completions++;
            }
        }
        writefln("completions=%d", completions);
    }   

    Thread consumer = new Thread(&Consumer).start;
    // produce NOP events
    foreach(i;0..iterations)
    {
        SubmissionEntry e;
        e.prepNop();
        e.user_data = i;
        while(io.full)
        {
            // short sleep while sq is full
            Thread.sleep(1.hnsecs);
        }
        io.put(e);
        io.submit();
        submissions++;
    }   
    writefln("submissions=%d", submissions);
    writefln("overflow=   %d", io.overflow);
    consumer.join();
}

See: https://github.com/axboe/liburing/blob/master/src/include/liburing.h#L76

The only field that can be set is the flags one but if I want to control the sq_thread_cpu I can't.

I think it would make sense to have a second setup function that will take the whole struct as an argument and copy it over for the setup.

First off, this is lovely. I have plans to integrate uring into the concurrency library and build servers based on that.

I'll probably have to support windows iocp too, and the static assert would be giving trouble.

Would you consider to refactor to just version(linux): ?

In any case, thanks for a good library.

liburing is conforming to memory ordering requirements of the kernel using it's hefty o_uring_smp._.. inline functions.
https://github.com/axboe/liburing/blob/master/src/queue.c#L223
https://github.com/axboe/liburing/blob/master/src/queue.c#L387
https://github.com/axboe/liburing/blob/master/src/queue.c#L53
https://github.com/axboe/liburing/blob/a209abe65befe603311b6fcd861e14d28e3e2f2d/src/include/liburing/barrier.h#L8
Didn't notice anything like it in during.

Thank you for writing such a nice interface to uring. I like the API a lot. However I cannot get it to work. Take this very simple example, adjusted from the readme. I should write "test" to a file opened with std.stdio : File:

#!/usr/bin/env dub
/+ dub.sdl:
	name "during-test"
    dependency "during" version="~>0.2.0"
+/
module examples.test_during;

import during;
import std.range : drop, iota;
import std.algorithm : copy, equal, map;
import std.stdio : File, writefln, writeln;
import std.string : representation;
import std.exception : enforce;
import std.array;

void main() 
{
    Uring io;
    auto res = io.setup();
    enforce(res >= 0, "Error initializing IO");
    
    File f = File("/tmp/test.txt", "wb");
    
    ubyte[] buffer = new ubyte[4];
    buffer[] = "test".representation[0 .. 4];
    writeln(buffer);
    SubmissionEntry entry;
    prepWrite(entry, f.fileno, buffer, 0);
    entry.user_data = 1;

    // chain operations
    res = io
        .put(entry) // whole entry as defined by io_uring
        .submit(1); // submit operations and wait for at least 1 completed

    enforce(res == 1); // 3 operations were submitted to the submission queue
    io.wait(1);
    enforce(!io.empty); // at least one operation has been completed
    
    foreach (ref CompletionEntry cqe; io) {
        writefln("user data=%s, result=%s", cqe.user_data, cqe.res);
    }
}

I get EFAULT (-14) as a result code for the operation. Do you have any pointers on what I am doing wrong?