Comments (7)
I remember having looked at this with lemoer. There was still a performance disadvantage compared to a in-kernel tunnel like WireGuard because of the copying process of the packets between user- and kernelland.
Is there a possibility to introduce something comparable to MSG_ZEROCOPY with io_uring? I've always wondered why Jason Donenfeld, the OpenVPN or tinc team didn't work on exposing the virtualization TAP sockets...
from fastd.
Hm... I just had a look at the current state of the code. SOCK_ZEROCOPY has already been implemented.
https://elixir.bootlin.com/linux/v5.8-rc3/source/drivers/net/tap.c#L693
So the only big performance difference between an in-kernel tunnel and an userland one is the additional copying of an skb to an iov. We can't get rid of that one, because skbs can't be forced to be in the user memory region. Actually I expected the impact of the copying to be much lower... By the way @lemoer we've already talked to NeoRaider about io_uring last year on IRC...
from fastd.
Here's a proof-of-concept patched fastd branch which lemoer and I created:
https://github.com/CodeFetch/fastd/tree/final
Indeed the syscall overhead can be reduced with io_uring. Unfortunately a kernel version >5.7 is required to allow poll-retry/fastpoll which is crucial for the performance gain.
Furthermore along a number of minor bugs some race conditions seem to occur unless the operations on an individual socket are being hardlinked. This patch works around this issue which introduces a slight performance penalty. It might have been fixed upstream already and needs further testing. I'll open up a pull-request when NeoRaider has reworked the buffer management to reduce the allocation overhead.
from fastd.
@NeoRaider are you done with the buffer pool? I've got a commit somewhere where I started to implement a dynamic buffer pool (which might grow if there is a high demand and shrinks when they are not needed anymore). It looks like your changes are compatible. A dynamic buffer pool is needed for getting a good performance for io_uring while keeping a low memory footprint.
BTW... What about introducing shared memory to implement threading support? Have you given it a thought already? I guess with io_uring the crypto performance will become the bottleneck. Is it possible to do the crypto with packets not "in order"? Otherwise I'd at least hope to make use of more cores on the servers with multiple slave processes.
from fastd.
The new buffer implementation is finished.
I don't understand the question about shared memory - threads always share their memory? Doing packet processing in threads should be fine on multi-core systems (but it will require some careful locking and/or barriers to ensure that no state is changed when the worker threads do not expect it).
I think packet processing for each peer should be serialized to avoid introducing additional reordering (fastd can handle packets reordered by up to 64 sequence numbers, but the transported network protocols may not), but as multi-core systems usually play a central role in a network and are connected to many peers, this could still provide some speedup.
from fastd.
@NeoRaider Sorry I meant subprocesses not pthreading - Shared memory between processes. Pthreads wouldn't bring a performance increase I guess, would they? Indeed I aim for making use of multiple cores, which isn't possible with pthreads only, is it?
from fastd.
Using multiple cores is the main use case of threads. In fact, the Linux kernel does not really distinguish between processes and threads - a thread is just a process that shares its PID, memory, file descriptors, and a few other things with its parent.
Using multiple processes as workers only makes sense when you need to isolate them from each other, for example to contain crashes or security issues. For fastd, multithreading is the way to go: It should be easier to implement for our use case and uses fewer resources (as almost all memory is shared).
from fastd.
Related Issues (20)
- fastd 20 osx build failure
- Broken init script? HOT 4
- Android still supported? HOT 4
- fastd 22 build issue HOT 3
- Explain what fastd is HOT 1
- Any chance to get encryption over l2tp offloading? HOT 5
- Interface value null with offloaded null@l2tp in socket response
- Establish hook not always called on connect HOT 4
- OpenRC service management script HOT 3
- arp flood if forward enabled HOT 4
- Multipath optin using MP-DCCP with tunprox HOT 1
- No packets received on fastd interface HOT 2
- Keepalive mechanism HOT 5
- Best MTU for L2TPv3 mode with kernel offload? HOT 1
- Add VXLAN backend HOT 1
- Distributed peering HOT 3
- Question: Does fastd implement perfect forward secrecy? HOT 2
- request a new release HOT 1
- Enhancement: config: allow setting a file-path rather then the raw secret HOT 2
- Question: What does `dropping duplicate packet from` mean? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastd.