rub-syssec / nyx-net Goto Github PK

Dockerfile 3.16% Shell 31.18% C 18.69% Python 46.98%

nyx-net's Introduction

Nyx-Net: Network Fuzzing with Incremental Snapshots

Nyx-Net is fast full-VM snapshot fuzzer for complex network based targets. It's built upon kAFL, Redqueen and Nyx. The approach is described in more detail in our paper.

Note:

This repository is serving only to capture the state as described in the paper. If you want to actually use Nyx, check out the maintained code released for the Nyx framework or it's Homepage.

@inproceedings{nyx-net,
author = {Schumilo, Sergej and Aschermann, Cornelius and Jemmett, Andrea and Abbasi, Ali and Holz, Thorsten}, 
title = {Nyx-Net: Network Fuzzing with Incremental Snapshots}, 
booktitle = {Proceedings of the Seventeenth European Conference on Computer Systems}, 
series = {EuroSys '22} 
year = {2022},
}

Setup

Please note that the minimum requirements to get Nyx-Net running are a recent linux kernel installed (>= v5.11) and full access to KVM. Fast-Snapshots and compile-time based tracing is supported by an unmodified vanilla kernel. If you want to fuzz closed-source targets with Nyx-Net's intel-PT mode, you will need to install KVM-Nyx.

The following commands will install all dependencies and setup Nyx-Net an your machine (assuming an Ubuntu 21.04):

git clone https://github.com/RUB-SysSec/nyx-net
cd nyx-net
./setup.sh

ProFuzzBench

To setup all ProfuzzBench targets you can use a Dockerfile which we provide with this repository. Using Docker is necessary because most of the targets will not compile on more recent linux distributions. This process will take several minutes to finish. Because of that, we provide two ZIP-archives (ASAN and non-ASAN), which contain a prepared collection of all targets.

To unpack the PFB builds from one of the archives, move to the targets folder and run either decompress_packed_targets.sh or decompress_packed_targets_asan.sh.

Optional: to compile and pack the targets using the Dockerfile move to the targets folder and then run docker_pack.sh. Once the this process has successfully finished, use the decompress_packed_targets.sh or decompress_packed_targets_asan.sh to finish the setup process.

Super Mario

Run the following script to compile and prepare the Super Mario Demo. This scripts expects the NES ROM of Super Mario Bros. (MD5: 811b027eaf99c2def7b933c5208636de) to be located at targets/extra_folders/super_mario_extra_folder/Super Mario Bros. (JU) (PRG0) [!].nes:

cd targets/
./prepare_super_mario.sh

This script will compile the x86 version of Super Mario Bros and create a Nyx sharedir for each level. The Nyx sharedirs will be put into the targets/packed_targets/ folder.

Run the Fuzzer

Move to the fuzzer/rust_fuzzer folder and run the following command to start the fuzzer:

cargo run --release -- \
      -s ../../targets/packed_targets/<TARGET>/

Incremental snapshots are disabled by default (Nyx-Net-None) but you can enable them by adding an -p parameter with one of the two available policies (balanced or aggressive). Parallel-fuzzing can also be enabled be using the -t <threads> parameter. Nyx-Net will automatically create a crash report once a crash of the target has been observed during fuzzing. Those reports are stored in the working directory (corpus/crash/*.log).

Reproducabilty

The fuzzer will store each new interesting input in the working directory. However, those inputs are stored in a specific bytecode format and cannot easily be used to replay the communication with a target running outside of Nyx. To turn those inputs into a more usable format, you need to re-run each finding in the corpus by another tool which is stored in fuzzer/rust_fuzzer_debug:

cargo run --release -- \
      -s ../../targets/packed_targets/<TARGET>/ \
      -d /tmp/workdir/corpus/crash/ \
      -t /tmp/workdir/corpus/crash_reproducible/

This tool will pick and re-execute each input form the crash folder and generate a more usable version which is then put into the crash_reproducible folder. To replay one of the files from the crash_reproducable folder, move to the packer/packer/ directory and use the nyx_net_payload_executor.py tool:

#supported modes: stdout, udp, tcp
python nyx_net_payload_executor.py /tmp/workdir/corpus/crash_reproducable/cnt_10.py stdout > /tmp/trace; hexdump -C /tmp/trace

ProfuzzBench

To reproduce the evaulation and results from the paper, you can use the scripts and tools located in profuzzbench-nyx/. More details on that can be found here.

License

The content of this repository is provided under AGPL license. However, this does only apply to this repository without any submodule. Please refer to each submodule from this repository to get more detailed information on each license used.

Free Software Hell Yeah!

Proudly provided by:

nyx-net's People

Contributors

Stargazers

Watchers

Forkers

kharos102 bb33bb nick0ve cybermonitor asdyxcyxc fatgrass alainlompo sucof by1c schumilo microsvuln tylzh97 ris3-lab hikame sohu0106 ol-imorozko killvxk humpty99 nemo6655 gmh5225

nyx-net's Issues

Network Emulation

I am quite interested in your work!
You said you hooked most socket API functions by LD_PRELOAD. Where can I find this code?

error[E0793]: reference to packed field is unaligned. Ubuntu 20.04

nyx-net git:(main) ./setup.sh
[?] Checking submodules ...
[?] Checking QEMU-NYX ...
[?] Checking NYX-Net fuzzer ...
[*] Compiling NYX-Net fuzzer ...
[?] Checking submodules ...
[?] Checking rust_fuzzer ...
Compiling fuzz_runner v0.1.0 (/home/ilorj/Desktop/protocols/nyx-net/fuzzer/libnyx/fuzz_runner)
error[E0793]: reference to packed field is unaligned
--> /home/ilorj/Desktop/protocols/nyx-net/fuzzer/libnyx/fuzz_runner/src/lib.rs:221:9
|
221 | &self.feedback_data.shared.ijon.max_data
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: fields of packed structs are not properly aligned, and creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
= help: copy the field contents to a local variable, or replace the reference with a raw pointer and use read_unaligned/write_unaligned (loads and stores via *p must be properly aligned even when using raw pointers)

For more information about this error, try rustc --explain E0793.
error: could not compile fuzz_runner due to previous error

nyx-net git:(main) cargo --version
cargo 1.69.0 (6e9a83356 2023-04-12)
nyx-net git:(main) rustc --version
rustc 1.69.0 (84c898d65 2023-04-16)

No vm.img

When I run the forked-daapd target, no vm.img was found in targets/packed_targets directory. So where can I find the vm image or how can I create that?

Comparison of Snapshot Taking Costs vs. Test Case Transmission Costs

I am very fond of your work on protocol fuzz testing, and I am currently attempting to optimize nyx-net. In nyx-net, the transmission of prefix messages is achieved through taking incremental snapshots.

I have a question: During the fuzz testing process, are there scenarios where the cost of directly sending prefix messages is less than the cost of taking incremental snapshots? Is this phenomenon common?

For instance, if there are 10 prefix messages, would it be faster to place a snapshot after the 8th message and then restore, followed by sending the remaining two messages to reach a certain state, rather than placing a snapshot directly after the 10th message?

How to debug target hanging inside the qemu-nyx instance started by rust_fuzzer?

Hi there,

We are trying to use nyx-net fuzzing some complex interactive systems, and we have successfully started the target system inside qemu-nyx manually. Unfortunately, the target came into hanging inside qemu-nyx instance when we started it with rust_fuzzer. According to the hook message printed by the code from your DEBUG_MODE and added by us, we found that our target is blocked somewhere and never come into socket() and bind(). As a result, it cannot reach the init_nyx() function and cannot start the fuzzing process.

We have tried several ways to debug. Specifically, we tried to use gdbserver to enable remote debugging, but it turns out that the qemu-nyx will kill the process started by gdbserver in the fork() hooked by ld_preload_fuzz.so. We have also tried connecting the rust_fuzzer-started qemu-nyx with ssh, but it seems the standard output of qemu-nyx has been captured by the chardev used for controlling fuzzing. Is there any possible way to debug our situation?

Some configs of concern

We have modified many configurations of qemu-nyx, here some critical ones:

We have raised the version of the kernel to 6.13 and started the qemu-nyx vm from a debian image.
Our target system runs in a multi-threaded mode. Many pthread_* functions are called when the target is running.

After using `afl-clang-fast` to instrument the program, I put it into the `target` folder and run it with `NYX-Net` `rust_fuzzer`. But the process failed.

Hello, can NYX-NET use afl-clang-fast? Or is there something wrong with my operation? Looking forward to your reply, thank you!

Few questions

Appreciate releasing source code for this significant work. I can see packer is doing some of the work when preparing the VM. Are there step by step instructions/tutorial on setting up a new target, creating agent, IPC spec for agent and host communication, using packer etc. Please point to any reference you can if there are no documentation currently. Thanks

HYPERCALL_KAFL_PANIC_EXTENDED: assert: setup_interpreter src/ld_preload_fuzz.c 325: offsets[0] == 1840245495844197195ULL

I use sudo sh libnyx/libnyx/tesh.sh (https://github.com/nyx-fuzz/libnyx/blob/main/libnyx/test.sh) command to fuzz nyx_tinydtls (obtained from https://github.com/RUB-SysSec/nyx-net/tree/main/targets/packed_targets). But the result nyx_exec (C interface nyx_exec in https://github.com/nyx-fuzz/libnyx/blob/34679b90dcdcc3c9834e4a510e6fc3ebb77b2246/libnyx/src/lib.rs) is always equal to 1. And I use nyx_print_aux_buffer (C interface in https://github.com/nyx-fuzz/libnyx/blob/34679b90dcdcc3c9834e4a510e6fc3ebb77b2246/libnyx/src/lib.rs) to get the result which shows HYPERCALL_KAFL_PANIC_EXTENDED: assert: setup_interpreter src/ld_preload_fuzz.c 325: offsets[0] == 1840245495844197195ULL. Here are the results snapshot. I don't know how to solve this problem. Looking forward your reply. Thank you!