Giter Club home page Giter Club logo

ruffles's Introduction

Ruffles

Build Status NuGet GitHub Release Discord Licence

Ruffles is a fully managed UDP library designed for high performance and low latency.

Why another reliable UDP library?

There are many RUDP libraries such as ENET, Lidgren, LiteNetLib. While many of them are great, Ruffles aims to fill in one niche that is largely not filled, that being lightweight fully managed libraries.

To compare to the examples above, ENET is amazing and is pretty much what Ruffles wants to be, but it's unmanaged. Lidgren, LiteNetLib and many other managed libraries can feel too bloated and contain many features that are unnecessary and they are often much slower.

Features

Ruffles has many features that other libs lack. See below for a summary and a detailed explanation of each.

  • Connection challenge
  • DOS amplification prevention
  • Slot filling prevention
  • Connection management
  • High performance and garbage free
  • Channeling
  • Threaded by default
  • Thread safe
  • Dependency free
  • IPv4 and IPv6 dual mode
  • Small packet merging
  • Fragmentation
  • Ack merging
  • Connection statistics
  • Path MTU discovery
  • Bandwidth tracking

Connection Challenge

The Ruffles protocol requires a challenge to be completed before a connection can be established. Currently, the challenge is a hashcash like challenge that is supplied by the server, brute force solved by the client and submitted. (Uses Fowler-Noll-Vo hash function instead of SHA1 currently).

DOS Amplification Prevention

DOS amplification is prevented by requiring unproportional connection message sizes. In addition, because of the connection challenge it's not computationally feasible to attack on Layer 4.

Slot Filling Prevention

Ruffles has a fixed amount of connection slots that can be used for pending connections, this limits the usability of slot filling attacks on Layer 4. Pending connections have a fixed timeout to solve the computationally expensive HashCash challenge before being knocked out, the slot will be available once again after that. As this only limits slot filling attacks, Ruffles also has a security mechanism where a HashCash has to be solved in the first message. This challenge is generated by the client and the server will verify that the date used is recent and that the IV has not already been used, forcing clients to recompute the HashCash challenge every time they want to initialize a handshake.

With these security mitigations, the only way to bring the server down is to exhaust all CPU resources.

Connection Management

Ruffles handles all connection management for you. It's a fully connection oriented protocol with heartbeat keepalive packets sent to ensure the connection is alive.

High Performance

Ruffles is fully garbage free, this is accomplished with a custom memory allocator in GC space. This ensures no memory is leaked to the garbage collector unless for resizing purposes. This makes Ruffles blazing fast. It also avoids memory copies as much as possible. Because Ruffles still runs in GC space, any memory leaks in Ruffles will be handled by the garbage collector and the user will be notified as the memory's destructor is called along with a stacktrace of where the leaked memory was originally allocated. See Implementation.

Reliability and Sequencing

There are currently a few ways of sending messages in Ruffles. The types are:

Reliable

All messages are guaranteed to be delivered, the order is not guaranteed, duplicates are dropped. Uses a fixed sliding window.

ReliableSequenced

All messages are guaranteed to be delivered with the order also being guaranteed, duplicates are dropped. Uses a fixed sliding window.

ReliableSequencedFragmented

All messages are guaranteed to be delivered with the order also being guaranteed, duplicates are dropped. Uses a fixed sliding window. Allows large messages to be fragmented.

Unreliable

Delivery is not guaranteed nor is the order. Duplicates are dropped.

UnreliableOrdered

Delivery is not guaranteed but the order is. Older packets and duplicates are dropped.

UnreliableRaw

Delivery is not guaranteed nor is the order. Duplicates are not dropped.

UnconnectedMessages

Raw UDP packets that does not require a connection.

ReliableOrdered

All messages are not guaranteed to be delivered. If you send multiple messages, at least one is guranteed to arrive. If you send a single message, it is guaranteed to arrive. Messages will always be in order. Duplicates are dropped.

ReliableFragmented

All messages are guaranteed to be delivered, the order is not guaranteed, duplicates are dropped. Uses a fixed sliding window. Allows large messages to be fragmented.

Threading

Ruffles is natively multi threaded and uses a background worker thread by default to handle network I/O.

Thread Safe

All public APIs in Ruffles are designed to be thread safe and can be accessed from any thread.

Dependency Free

Ruffles is 100% dependency free, it's thus very portable and should run on most platforms.

IPv6 Dual Mode

Ruffles supports IPv6 dual socket mode. It does this by using two sockets bound to the same port, thus accomplishing full dual stack functionality that is invisible to the user.

Small Packet Merging

Small packets will be delayed for sending, this allows them to be merged into one larger packet. This can be disabled and enabled on a per packet basis. The delay and max merge size can also be configured.

Fragmentation

Packets can be sent as ReliableSequencedFragmented or ReliableFragmented which allows for a single packet to be of a size of up to 2^15*1450 bytes = 47513600 bytes = 47.5 megabyte.

Ack Merging

Ack packets are merged into bitfields to make them much more compact.

Connection Statistics

Detailed statistics can be retrieved from connections, including the bytes sent, packets sent, round trip times and more.

Path MTU

Automatically discovers the largest MTU possible for each connection.

Bandwidth Tracking

Limit the amount of traffic allowed to be sent to a connection. Custom algorithms can be adapted with the IBandwidthTracker interface.

Roadmap

This is stuff I want to and plan to add

  • Adaptable window sizes
  • Basic bandwidth control, limit the amount of acks that are sent etc
  • More Fragmentation Types
  • Explicit Nack
  • MLAPI.Relay Support
  • MLAPI.NAT (Holepuncher) support
  • Bloatless Moduled Library (Make all the garbage features like relay support separate modules to keep the core library bloat free and small)

Maybe Roadmap

Here are the features that are considered but not decided. This is to prevent bloat.

  • Multicasting
  • Meshing / Peer relaying

Fragmented

The fragmented channel currently does not have any flow rate for ack resends.

Unity Support

Due to a Unity bug, Ruffles does not work properly in IL2CPP by default. The bug has been reported to Unity. If you need to run Ruffles with IL2CPP, compile it with the MILLISECONDS_SELECT define. This will make the Socket.Select method use a millisecond based timeout instead of microseconds. This has been patched. Feel free to use IL2CPP in your project.

ruffles's People

Contributors

gisforgravity avatar twotenpvp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ruffles's Issues

Why I get AckNotification for unreliable channel?

Describe the bug
When i send an unreliable packet, I receive an AckNotification for that packet. But i don't want to get an ack.

To Reproduce
Steps to reproduce the behavior:

  1. Send an Unreliable or UnreliableOrdered packet.
  2. Check for AckNotification event.

Expected behavior
There should be no Ack.

Environment (please complete the following information):

  • OS: Win11 and Manjaro
  • Unity Version: 2022.3.6f1
  • Ruffles Version: v11.1.9

Issue with iOS sockets resuming after screen is locked

When using Ruffles as transport for MLAPI v11, when iOS client device has screen locked for over ~5 seconds, after the device is unlocked Ruffles library begins spamming error messages and the transport stops working entirely.

Error includes this exception in printout:
System.Net.Sockets.SocketException (0x80004005): The socket is not connected

Error message is being printed in the Socket's StartSocketLogic loop.

if (Logging.CurrentLogLevel <= LogLevel.Error) Logging.LogError("Error when receiving from socket: " + e);

This issue appears to be identical to issue that is reported to LiteNetLib repo here:
RevenantX/LiteNetLib#312

For reference on solution, LiteNetLib fixed this issue in the following commits:
RevenantX/LiteNetLib@6142768
RevenantX/LiteNetLib@dac4fea

image

Hashcash should use stronger hash

Describe the bug
FNV is not suitable for Hashcash.

return (difficulty == 0 || ((HashProvider.GetStableHash64(challenge + additionsRequired) << ((sizeof(ulong) * 8) - difficulty)) >> ((sizeof(ulong) * 8) - difficulty)) == 0);

Any math undergrad can break this in his spare time.
You need a stronger cryptographic algorithm for hashcash. The state of the art appears to be sha256, or at least sha1:
https://en.bitcoin.it/wiki/Hashcash#Hash_function_choices

Calling GetCurrentRtt with invalid clientId throws exception

When calling GetCurrentRtt with an invalid clientId causes an exception two different ways:

KeyNotFoundException: The given key was not present in the dictionary.
System.Collections.Generic.Dictionary`2[TKey,TValue].get_Item (TKey key) (at :0)
RufflesTransport.RufflesTransport.GetCurrentRtt (System.UInt64 clientId) (at C:/projects/mlapi-transports/RufflesTransport/RufflesTransport.cs:281)

NullReferenceException: Object reference not set to an instance of an object
RufflesTransport.RufflesTransport.GetRufflesConnectionDetails (System.UInt64 clientId, System.UInt64& connectionId) (at C:/projects/mlapi-transports/RufflesTransport/RufflesTransport.cs:332)
RufflesTransport.RufflesTransport.GetCurrentRtt (System.UInt64 clientId) (at C:/projects/mlapi-transports/RufflesTransport/RufflesTransport.cs:279)

Could GetCurrentRtt simply not throw an exception but rather return 0?

Ack packets merging

Hi and big thanks for a great library!
Could you explain how the ack packets are merged into bitfields? I'm trying to understand the method behind it all...

Forward Error Correction

A nice feature would be to introduce FEC on messages that are fragmented, trading bandwidth for reliability. This is specially useful on scenarios where clients are far from a streaming server.
Reed-Solomon would be enough, a message could be reconstructed using 2 out of 3 packets, 3 out of 4, etc, avoiding retransmition in most cases, saving a lot of round trips.

New Channel: UnreliableSequenced

Is your feature request related to a problem? Please describe.
The problem with UnreliableOrdered is that it drops older messages even if the game didn't read the pending ones yet. This is a waste of information that is useful to the game and unnecessary packet loss.

Describe the solution you'd like
The solution is to implement a UnreliableSequenced channel that keeps older messages and sorts them until the game/MLAPI performs a read on them. After the read, older packets are dropped.

Additional context
A good example of this in practice is this post from the Lead Networking Programmer of Epic Games where they had to implement this in Fortnite to fix the packet loss problem:
https://www.reddit.com/r/FortNiteBR/comments/awagpo/packet_reordering_technical_post/

Async

Any future planes to implement an async version as opposed to manual threads?

SendTo sync with main thread

Is your feature request related to a problem? Please describe.
Can SendTo be triggered by the main thread? i have a game loop on my game server and want to send all messages at the end of every step
Describe the solution you'd like
Main Thread trigger Event at end of loop, SendTo Thread "WaitsFor" Event to run logic

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

RuffleSocket interface

RuffleSocket's public behavior should be exposed as an interface (ISocket) to allow for easier unit-testing (i.e. by mocking implementation)

Upm support

Is your feature request related to a problem? Please describe.
Keeping assets up to date in unity is extremely painful. It pollutes your project file structure. If you upgrade, it doesn't delete old files. There is no way to tell which version you have installed.

Describe the solution you'd like
Unity came up with a much better way to use assets: UPM. Please provide a UPM package.json file in your repo so that I can install ruffles using UPM.

See here for some documentation:
https://medium.com/openupm/how-to-maintain-upm-package-part-1-7b4daf88d4c4

See here for MirrorNG, which does provide UPM distribution:
https://github.com/MirrorNG/MirrorNG

Additional context
My goal is to make a MirrorNG transport for ruffles, I would like to add ruffles as a UPM dependency so that I don't have to maintain a copy of your files.

Simulating latency not working.

Describe the bug
Using the simulator in MLAPI and ruffles as a transport.
Setting latency to 1800-2000ms has no effect and latency is still 0ms.
To Reproduce
Steps to reproduce the behavior:

  1. Create MLAPI project
  2. Use ruffles
  3. Enable Simulator with latency
  4. Test latency
  5. 0 ms

Expected behavior

There should be a delay of 1800-2000ms.

Environment (please complete the following information):

  • OS: Windows 10
  • Unity Version: 2019.1
  • Ruffles Version: LTS from MLAPI installer

Build against Core 3.1

I see the project is still actively being developed and updated as of 20 days ago... Is there any chance a build against .net core 3.1 could be supplied officially too? I have no troubles building it myself to get that now, but it would be nice since this is still maintained to have nuget packages against the core version I'm using to get clean/quick updates to the latest build as I'm starting a project that will possibly have a community maintaining it in the future. As a spoiler to what that project will be, It'll be a C# based project for modding N64/PS1 games with built in network functionality (This for my own mods in mind being online multiplayer based).

Connection after network switch

After network switch client's IP can change. TCP connection will die after this. And it looks like Ruffles connection will die too. So to keep player connected i have to reconnect and keep my own queue of delivered messages over reliable channel.

Is there any way to keep connection alive after IP change? Or some way to copy reliable channels data to new connection?

Or may be you know if any other library support such feature?

Ruffles transport: client can't connect

Describe the bug
Client can not connect to a host using Ruffles Transport. Error message is "Connection not alive". Firewall on host is disabled, netstat shows port is being listened

To Reproduce
Simple project using Ruffles transport

Expected behavior
Client connects successfully

Environment (please complete the following information):

OS: Windows 10 & Oculus Quest (Android)
Unity Version: 2019.1.6f1
MLAPI Version: 10.13.0
Ruffles: No version shown in Unity, Ruffles.dll says 1.0.0

Leaving Ruffles running for a certain period of time results in disconnects and exceptions

Hi,

I'm currently testing ruffles, and before I say anything, you did amazing!

Currently, I try a small simulation of four RuffleSockets, one being the server and three the clients. Each 16 to 20ms all participating sockets send a 256-byte package. The server "broadcasts" the package to the three clients. The clients just send it directly to the server. Each socket creates its own packet, so they are not dependent on any information flow.

After about ~365,000 messages in total (~54000 received payloads on each client, and ~196000 payloads received on the server), each client receives a NetworkEvent with NetworkEventType.Disconnect.

The Disconnects are a couple of seconds apart (roughly the same time I need to press connect on each client). 13 seconds later the server throws a NullReferenceException and then logs that a particular client has disconnected.

NullReferenceException: Object reference not set to an instance of an object Ruffles.Memory.MemoryManager.DeAlloc (Ruffles.Memory.HeapPointers pointers) (at <f3d7047ed8d941c398bcc2452492ea15>:0) Ruffles.Messaging.PacketHandler.SendMessage (System.ArraySegment1[T] payload, Ruffles.Connections.Connection connection, System.Byte channelId, System.Boolean noDelay, Ruffles.Memory.MemoryManager memoryManager) (at :0)
Ruffles.Core.RuffleSocket.Send (System.ArraySegment1[T] payload, Ruffles.Connections.Connection connection, System.Byte channelId, System.Boolean noDelay) (at <f3d7047ed8d941c398bcc2452492ea15>:0)

All this is run inside Unity 2018.4.9f1. It's a simple repo, I can provide access if needed.
Unfortunately, I have no idea why the client suddenly thinks it is disconnected.

The Editor.log from Unity contains a bit of more information:

ClientEvent: Disconnect
[WARNING] Client 123.123.123.123:5674 connection could not be found
The warning is issued without a stack trace.

The server crash seems to be related to:

[ERROR] Outgoing packet window is exhausted. Disconnecting

This probably happens because I try to send via the ChannelType.Reliable channel.

I hope that helps to solve the issue, and or maybe you have ideas on how I can fix it on my side :)

Add functionality for graceful disconnects with Fragmented channels

When a server disconnects a connection that has ReliableSequencedFragmented as the channel type (and potentially others), any pending packets are not sent to the client. This results in "friendly disconnects" (use case - user is on the wrong version of the client, needs a message telling them to upgrade, then disconnect them) not being received by the client, since the _pendingSends aren't flushed to the socket, they're simply reclaimed. See ReliableSequencedFragmented.Release().

I believe I understand the motivation - the Disconnect event should disconnect immediately and not dump data to clients - but there isn't a mechanism to flush() partial packets to users. Unfortunately, I think alternative approaches are not good (ask the client to disconnect in the "final packet" and trust them to do so; mark a connection as pending disconnect and complete the disconnect after MergeDelay amount of time; or my solution - expose a second Send() method with noMerge=true for these scenarios).

I'm not sure if this repo is being actively developed any further with the maintainer moving over into other unity networking solutions, but if the maintainer (or anyone else capable) would like to, this would be a nice feature, both for this use case and others.

Proposal 1: Add Flush() to IChannel and implement in the various channel implementations
Proposal 2: Expose method for polling count of _pendingSends in the channels.
Proposal 3: Add another Disconnect() method or parameter that attempts to Flush() pending/unmerged packets to the client.

Thanks for the consideration

Ruffles logic loop optimization

The current Ruffles logic loop iterates the the whole Connections array about 7 times. Performing null and dead checks for each.

This could be reduced by either pre-caching the connected connections, or exiting once the last connected is reached.

It could be further reduced by merging the loops and nullchecks into a single loop.

Deadlock in ReliableSequencedChannel.HandleAck(ArraySegment<byte> payload)

Describe the bug
Endless loop due to wrong data type in counter variable i.

To Reproduce
I don't have enough understanding of the inner workings to tell how exactly to reproduce this. However it's obvious that the for loop in ReliableSequencedChannel.HandleAck(ArraySegment<byte> payload) can never complete when bits is greater than 255. Not sure IF bits should ever be greater than 255, but it is in my case. The debugger shows a value of 4128 in my case.
Same issue may probably occur in ReliableChannel as well.

Expected behavior
It shouldn't lock up. Either the counter i should be of type int or it should be guaranteed that bits can never exceed 255.

Environment (please complete the following information):

  • OS: Debian 11
  • Unity Version: N/A (this is a pure .NET 7 application)
  • Ruffles Version: 11.1.9
  • Ruffles Commit: aab94e4

Additional context
The deadlock occurs after a few minutes during loadtests of my game server. It is possible that the game server is sending or receiving too much data which may increase the chance of this issue ocurring.

how to use this in unity?

I couldn't find any documentation regarding sending information reliably or unreliably, and it's unclear whether packet loss is checked. There are many questions due to the absence of documentation.

Rewrite sliding set

I believe there is big potential for performance upgrades to the SlidingSet. It's not critical as it's only used for validating challenge initialization vectors. But should be investigated.

Reuse channels

Ruffles currently reuses everything (including memory, connections and more). It currently does not reuse channels which can contain huge amounts of memory. (Though their memory is usually reused except for collections)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.