netsys / netbricks Goto Github PK

NetBricks: A new network function framework based on Rust.

License: ISC License

C 6.25% Makefile 0.66% C++ 0.32% Rust 67.65% Python 1.34% Shell 9.90% LLVM 13.44% Dockerfile 0.44%

netbricks's Introduction

NetBricks is a Rust based framework for NFV development. Please refer to the paper for information about the architecture and design. Currently NetBricks requires a relatively modern Linux version.

Building

NetBricks can be built either using a Rust nightly build or using Rust built from the current Git head. In the later case we also build musl and statically link to things. Below we provide basic instructions for both.

Finally, in addition to the above options, NetBricks can also be built within a Docker container. In this case, you do not need to install any of the dependencies, and the final product can be run the same. However to run NetBricks you still need to be on a machine that is correctly configured to run DPDK (see here for more details), and you still need to install Rust nightly (for libraries). Please see the container build instructions for more information.

Using Rust Nightly

First obtain Rust nightly. I use rustup, in which case the following works

curl https://sh.rustup.rs -sSf | sh  # Install rustup
source $HOME/.cargo/env
rustup install nightly
rustup default nightly

Then clone this repository and run build.sh

./build.sh

This should download DPDK, and build all of NetBricks.

Using Rust from Git

The instructions for doing so are simple, however building takes significantly longer in this case (and consumes tons of memory), so do this only if you have lots of time and memory. Building is as simple as

export RUST_STATIC=1
./build.sh

Dependencies

Building NetBricks requires the following dependency packages (on Debian):

apt-get install libgnutls30 libgnutls-openssl-dev libcurl4-gnutls-dev libnuma-dev libpcap-dev

NetBricks also supports using SCTP as a control protocol. SCTP support requires the use of libsctp (this is an optional dependency) which can be installed on Debian using:

apt-get install libsctp-dev

Tuning

Changing some Linux parameters, including disabling C-State, and P-State; and isolating CPUs can greatly benefit NF performance. In addition to these boot-time settings, runtime settings (e.g., disabling uncore frequency scaling and setting the appropriate flags for Linux power management QoS) can greatly improve performance. The energy.sh in scripts/tuning will set these parameter appropriately, and it is recommended you run this before running the system.

Container Build

You must have Docker installed. Once this is done, just run

./build.sh build_container

This will build and copy the binaries over to the target subdirectory. As noted above, you can run it if you have a DPDK compatible machine.

Example NFs

This repository includes a set of example NFs under the test directory. A complete list of example can be found by running

./build.sh run

The build script can be used to run these examples as

./build.sh run <example name> <options>

Passing -h will provide a list of options. All of these examples accept one or more ports as input. Ports can be specified as one of:

PCI ID of a NIC
dpdk:<PMD spec> where PMD spec can be something like dpdk:eth_pcap0,rx_pcap=$HOME/tcpflow/tests/udp.pcap,tx_pcap=out.pcap which specifies a PCAP file should be used. See DPDK source for other PMD drivers that are available.
ovs:<integer> to connect to an OpenVSwitch DPDK ring port (dpdkr).
bess:<port name> to connect to a BESS ZeroCopyVPort

Future Work

Support for futures for control plane functionality.

netbricks's People

Contributors

Stargazers

Watchers

netbricks's Issues

compilation errors on build

on Ubuntu 18.04 LTS, using Rust nightly,

error[E0588]: packed type cannot transitively contain a `[repr(align)]` type
  --> framework/src/shared_state/directory.rs:13:1
   |
13 | / pub struct DirectoryHeader {
14 | |     entries: AtomicUsize,
15 | |     // Used to signal that snapshotting is in progress.
16 | |     current_version: AtomicUsize,
17 | |     committed_version: AtomicUsize,
18 | |     length: usize,
19 | | }
   | |_^

unit test failure?

I just get this one (the rest pass) while running netbricks on a container:


---- native_include::bindgen_test_layout_max_align_t stdout ----
	thread 'native_include::bindgen_test_layout_max_align_t' panicked at 'assertion failed: `(left == right)`
  left: `24`,
 right: `32`: Size of: max_align_t', /opt/netbricks/target/release/build/e2d2-8829eb254440fd22/out/dpdk_bindings.rs:1535:438
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at libstd/sys_common/backtrace.rs:59
             at libstd/panicking.rs:380
   3: std::panicking::default_hook
             at libstd/panicking.rs:390
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:576
   5: std::panicking::begin_panic
             at libstd/panicking.rs:537
   6: std::panicking::begin_panic_fmt
             at libstd/panicking.rs:521
   7: e2d2::native_include::bindgen_test_layout_max_align_t
   8: <F as alloc::boxed::FnBox<A>>::call_box
             at libtest/lib.rs:1462
             at /checkout/src/libcore/ops/function.rs:223
             at /checkout/src/liballoc/boxed.rs:788
   9: __rust_maybe_catch_panic
             at libpanic_unwind/lib.rs:102```

NetBricks integration with Tokio-rs

I am looking for alternate to Seastar in Rust lang and seems NetBricks can be used for developing high-performance network applications.
Any plans to integrate with tokia-rs?

UDF and Packetize

Hi all,

Just a question. I'm trying to understand the inner working of the window stream UDF combined with the packetize abstraction to modify flows on the fly, as suggested in the paper. Is there an example of this? How will the re-packetization will be made regarding, eg. SEQ and ACKs? Will it re-do a 3-way handshake etc?
One example would be rewriting URLs in HTTP flows to use different CDNs.

Thanks for the light,
Tom

unresolved import `std::alloc::Opaque`

I am installing an old RUST build which has a few dependencies. While compiling e2d2 this is the error i get. Can i get around this error without moving to any other build?

Edit: This occurs on the latest build too.

error[E0432]: unresolved import std::alloc::Opaque
--> framework/src/allocators/cache_aligned.rs:1:47
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^ no Opaque in alloc

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:2:1
|
2 | #![feature(asm)]
| ^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:3:1
|
3 | #![feature(log_syntax)]
| ^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:4:1
|
4 | #![feature(box_syntax)]
| ^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:5:1
|
5 | #![feature(specialization)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:6:1
|
6 | #![feature(slice_concat_ext)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:7:1
|
7 | #![feature(fnbox)]
| ^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:8:1
|
8 | #![feature(alloc)]
| ^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:9:1
|
9 | #![feature(heap_api)]
| ^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:10:1
|
10 | #![feature(unique)]
| ^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:11:1
|
11 | #![feature(const_fn)]
| ^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:13:1
|
13 | #![feature(ptr_internals)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:15:1
|
15 | #![feature(allocator_api)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0554]: #![feature] may not be used on the stable release channel
--> framework/src/lib.rs:17:1
|
17 | #![feature(integer_atomics)]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to 14 previous errors

Build fails in all three ways

When directly building by ./build.sh, it finally shows:

When building with RUST_STATIC=1, it shows:

When building in containers, it shows:

I am new to Rust, so I have not got any idea of how to fix them. Just want to run the NF successfully; any suggestions are highly appreciated!

missing field `dpdk_args` in initializer of `config::NetbricksConfiguration`

I am following the commands given in the readme file and I am getting the following error when I run

./build.sh build

_### error[E0063]: missing field dpdk_args in initializer of config::NetbricksConfiguration
_--> src/config/config_reader.rs:290:8
|
290 | Ok(NetbricksConfiguration {
| ^^^^^^^^^^^^^^^^^^^^^^ missing dpdk_args

error: aborting due to previous error

error: Could not compile e2d2.

To learn more, run the command again with --verbose.__

Am I missing something? Please help me in this regards.

Thanks.

when I am compiling e2d2 , there are some errors,

I use clang 3.9 on Ubuntu, when compiling e2d2 , there are errors like following:

SCTP library found
~/NetBricks/framework ~/NetBricks
Compiling e2d2 v0.2.0 (file:///home/luo/NetBricks/framework)
error[E0432]: unresolved import std::alloc::Alloc
--> framework/src/allocators/cache_aligned.rs:1:24
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^ no Alloc in alloc

error[E0432]: unresolved import std::alloc::Global
--> framework/src/allocators/cache_aligned.rs:1:31
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^ no Global in alloc

error[E0432]: unresolved import std::alloc::Layout
--> framework/src/allocators/cache_aligned.rs:1:39
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^ no Layout in alloc

error[E0432]: unresolved import std::alloc::Opaque
--> framework/src/allocators/cache_aligned.rs:1:47
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^ no Opaque in alloc

error[E0603]: module alloc is private
--> framework/src/allocators/cache_aligned.rs:1:18
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^

error[E0603]: module alloc is private
--> framework/src/allocators/cache_aligned.rs:1:24
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^

error[E0603]: module alloc is private
--> framework/src/allocators/cache_aligned.rs:1:31
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^

error[E0603]: module alloc is private
--> framework/src/allocators/cache_aligned.rs:1:39
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^

error[E0603]: module alloc is private
--> framework/src/allocators/cache_aligned.rs:1:47
|
1 | use std::alloc::{self, Alloc, Global, Layout, Opaque};
| ^^^^^^