Cloudtoid Interprocess is a cross-platform shared memory queue for fast communication between processes (Interprocess Communication or IPC). It uses a shared memory-mapped file for extremely fast and efficient communication between processes and it is used internally by Microsoft.
- Fast: It is extremely fast.
- Cross-platform: It supports Windows, and Unix-based operating systems such as Linux, MacOS, and FreeBSD.
- API: Provides a simple and intuitive API to enqueue/send and dequeue/receive messages.
- Multiple publishers and subscribers: It supports multiple publishers and subscribers to a shared queue.
- Efficient: Sending and receiving messages is almost heap memory allocation free reducing garbage collections.
- Developer: Developed by a guy at Microsoft.
The NuGet package for this library is published here.
Note: To improve performance, this library only supports 64-bit CLR with 64-bit processor architectures. Attempting to use this library on 32-bit processors, 32-bit operating systems, or on WOW64 may throw a
NotSupportedException.
This library supports .NET Core 3.1+ and .NET 6+. It is optimized for .NET dependency injection but can also be used without DI.
Creating a message queue factory:
var factory = new QueueFactory();Creating a message queue publisher:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);Creating a message queue subscriber:
options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);Adding the queue factory to the DI container:
services
.AddInterprocessQueue() // adding the queue related components
.AddLogging(); // optionally, we can enable loggingCreating a message queue publisher using an instance of IQueueFactory retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var publisher = factory.CreatePublisher(options);
publisher.TryEnqueue(message);Creating a message queue subscriber using an instance of IQueueFactory retrieved from the DI container:
var options = new QueueOptions(
queueName: "my-queue",
bytesCapacity: 1024 * 1024);
using var subscriber = factory.CreateSubscriber(options);
subscriber.TryDequeue(messageBuffer, cancellationToken, out var message);To see a sample implementation of a publisher and a subscriber process, try out the following two projects. You can run them side by side and see them in action:
Please note that you can start multiple publishers and subscribers sending and receiving messages to and from the same message queue.
A lot has gone into optimizing the implementation of this library. For instance, it is mostly heap-memory allocation free, reducing the need for garbage collection induced pauses.
Summary: A full enqueue followed by a dequeue takes ~250 ns on Linux, ~650 ns on MacOS, and ~300 ns on Windows.
Details: To benchmark the performance and memory usage, we use BenchmarkDotNet and perform the following runs:
| Method | Description |
|---|---|
| Message enqueue | Benchmarks the performance of enqueuing a message. |
| Message enqueue and dequeue | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message. |
| Message enqueue and dequeue - no message buffer | Benchmarks the performance of sending a message to a client and receiving that message. It is inclusive of the duration to enqueue and dequeue a message and memory allocation for the received message. |
You can replicate the results by running the following command:
dotnet run Interprocess.Benchmark.csproj -c ReleaseYou can also be explicit about the .NET SDK and Runtime(s) versions:
dotnet run Interprocess.Benchmark.csproj -c Release -f net7.0 --runtimes net7.0 net6.0 netcoreapp3.1Host:
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.22000
Intel Core i9-10900X CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK=6.0.201
[Host] : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
.NET 6.0 : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
Results:
| Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
|---|---|---|---|---|
| Message enqueue | 192.7 |
3.61 |
3.21 |
- |
| Message enqueue and dequeue | 305.6 |
5.96 |
6.62 |
- |
| Message enqueue and dequeue - no message buffer | 311.5 |
5.90 |
9.85 |
32 B |
Host:
BenchmarkDotNet=v0.13.1, OS=macOS Big Sur 11.6 (20G165) [Darwin 20.6.0]
Intel Core i5-8279U CPU 2.40GHz (Coffee Lake), 1 CPU, 8 logical and 4 physical cores
.NET SDK=5.0.401
[Host] : .NET 5.0.10 (5.0.1021.41214), X64 RyuJIT
.NET 5.0 : .NET 5.0.10 (5.0.1021.41214), X64 RyuJIT
Results:
| Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
|---|---|---|---|---|
| Message enqueue | 487.50 |
4.75 |
3.96 |
- |
| Message enqueue and dequeue | 666.10 |
10.91 |
10.20 |
- |
| Message enqueue and dequeue - no message buffer | 689.33 |
13.38 |
15.41 |
32 B |
On Ubuntu (through WSL)
Host:
BenchmarkDotNet=v0.13.2, OS=ubuntu 20.04
Intel Core i9-10900X CPU 3.70GHz, 1 CPU, 20 logical and 10 physical cores
.NET SDK=6.0.403
[Host] : .NET 6.0.11 (6.0.1122.52304), X64 RyuJIT AVX2
.NET 6.0 : .NET 6.0.11 (6.0.1122.52304), X64 RyuJIT AVX2
Results:
| Method | Mean (ns) | Error (ns) | StdDev (ns) | Allocated |
|---|---|---|---|---|
| Message enqueue | 5.3 |
- |
- |
- |
| Message enqueue and dequeue | 169.9 |
3.08 |
4.01 |
- |
| Message enqueue and dequeue - no message buffer | 179.4 |
1.91 |
1.60 |
32 B |
This library relies on Named Semaphores To signal the existence of a new message to all message subscribers and to do it across process boundaries. Named semaphores are synchronization constructs accessible across processes.
.NET Core 3.1 and .NET 6/7 do not support named semaphores on Unix-based OSs (Linux, macOS, etc.). Instead we are using P/Invoke and relying on operating system's POSIX semaphore implementation. (Linux and MacOS implementations).
This implementation will be replaced with System.Threading.Semaphore once .NET adds support for named semaphores on all platforms.
- Create a branch from
main. - Ensure that all tests pass on Windows, Linux, and MacOS.
- Keep the code coverage number above 80% by adding new tests or modifying the existing tests.
- Send a pull request.
Pedram Rezaei is a software architect at Microsoft with years of experience building highly scalable and reliable cloud-native applications for Microsoft.
Here are a couple of items that we are working on.
- Create a marketing/documentation website
- Once .NET supports named semaphores on Linux, then start using them.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent