Giter Club home page Giter Club logo

rakelimit's Introduction

Rakelimit

A multi-dimensional fair-share rate limiter in BPF, designed for UDP. The algorithm is based on Hierarchical Heavy Hitters, and ensures that no party can exceed a certain rate of packets. For more information please take a look at our blog post.

Usage

To activate rakelimit create a new instance and provide a file descriptor and a rate limit that you think the service in question won't be able to handle anymore:

conn, err := net.ListenPacket("udp4", "127.0.0.1:0")
if err != nil {
    tb.Fatal("Can't listen:", err)
}
udpConn := conn.(*net.UDPConn)

// We don't want to allow anyone to use more than 128 packets per second
ppsPerSecond := 128
rake, err := New(udpConn, ppsPerSecond)
defer rake.Close()
// rate limiter stays active even after closing

That's all! The library now enforces rate limits on incoming packets, and it happens within the kernel.

Requirements

The library should be go-gettable, and has been tested on Linux 5.11.

You may have to increase optmem_max depending on your distribution:

sudo sysctl -w net.core.optmem_max=22528

You will need a clang-12 binary if you want to recompile the filter. Simply run go generate in the root of the project.

Limitations

  • IPv6 doesn't support options
  • requires tweaking of optmem
  • not tested in production

Testing

go test .

rakelimit's People

Contributors

lmb avatar sauercrowd avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rakelimit's Issues

replace bpf_ktime_get_ns with faster bpf_jiffies64

The single call to bpf_ktime_get_ns uses about 30% of benchmark time on my machine. It would be a big perf win if we can replace it with something else.

There is a function bpf_jiffies64 which is probably cheaper, but less accurate. Open questions:

  • What is the resolution we can get from jiffies64, and is that enough for our purposes?
  • How do we convert jiffies64 to a timestamp?

compress countmin state to 64bits

The value we store in a count-min sketch is currently 128bit:

struct cm_value {
        fpoint value;
        __u64 ts;
};

I think we can compress this to 64 bits total: a __u32 for the estimated rate in pps and a __u32 for the timestamp. This means the timestamp would wrap every 4.2s but given our current WINDOW that isn't a problem. It just needs code to deal with it.

Doing this will halve the memory we need, plus on 64bit arches we can (probably) use a single load and store for struct cm_value which reduces the likelihood of observing a race condition.

Specialise BPF code for IPv4 & IPv6 to optimise performance

We currently map IPv4 addresses to IPv6 ones, and then work on IPv6 addresses exclusively. This is a significant performance hit, since we need to hash 4x more bytes (roughly).

If we split the code into IPv4 and IPv6 specific, we can determine the protocol in New and then attach the correct BPF filter. This should give us a nice boost for IPv4.

BPF exceeds default net.core.optmem_max limit

If you try to run tests / use the filter on stock Ubuntu, you get the following error:

$ sudo -E go test 
--- FAIL: TestNew (0.08s)
    rakelimit_test.go:14: Can't create limiter: can't attach BPF to socket: cannot allocate memory

This is because SO_ATTACH_BPF checks the size of the program against net.core.optmem_max limit. The default value for that on my Ubuntu install is 20480. We currently need something > 32768 but < 65536.

We should try to get the filter size below the default value. See also #2.

endianess of ip address data from __sk_buff

Hello,
When I first looked at the mask below (0xffffff00), I thought that it would incorrectly mask the MSB because I thought that the IP address would be in network byte order but when I printed the IP address, I realized that it was masking off the right byte but the address was backwards (1.0.0.127 instead of 127.0.0.1 for instance). It would seem that the address is in little endian (which in my case is the host endianness) given the mask and the bytes being masked off (after the mask I get 0.0.0.127). Why isn't the IP address in network byte order?

static FORCE_INLINE void ipv4_hash(struct in_addr ip, struct address_hash *a, struct address_hash *b)
{
        bpf_printk("IP: %pI4", &ip.s_addr);
	
        a->vals[ADDRESS_IP] = fasthash64(&ip, sizeof(ip), FH_SEED);
	b->vals[ADDRESS_IP] = hashlittle(&ip, sizeof(ip), L3_SEED);
	ip.s_addr &= 0xffffff00;
	a->vals[ADDRESS_NET] = fasthash64(&ip, sizeof(ip), FH_SEED);
	b->vals[ADDRESS_NET] = hashlittle(&ip, sizeof(ip), L3_SEED);
}

I also noticed that __sk_buff has a some ip address fields. Couldn't we use those fields instead of load_word and load_ipv6?

...
	__u32 remote_ip4;	/* Stored in network byte order */
	__u32 local_ip4;	/* Stored in network byte order */
...

Question about tools

What set of tools did you use to obtain the data for your graphs, like the one below? I haven't been able to get data for "traffic forwarded".

Screenshot from 2022-01-05 12-45-53

Also, what tool did you use to simulate this flood attack where the pps rate increases? I would like to run some tests, do some experiments and have data to compare.
Thank you for making your solution and code open source! It is very educational.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.