foniod / foniod Goto Github PK

View Code? Open in Web Editor NEW

412.0 19.0 33.0 3.47 MB

Data first monitoring agent using (e)BPF, built on RedBPF

License: GNU General Public License v3.0

Rust 91.15% Shell 2.82% Dockerfile 0.16% Cap'n Proto 0.20% HCL 4.49% Jinja 1.18%

rust agent monitoring ebpf statsd-client

foniod's Introduction

ingraind

Data-first Monitoring

ingraind is a security monitoring agent built around RedBPF for complex containerized environments and endpoints. The ingraind agent uses eBPF probes to provide safe and performant instrumentation for any Linux-based environment.

InGrain provides oversight of assets and risks:

Your customer data - an employee copying your customer database to their personal cloud store.
Your infrastructure - an attacker executing a zero day attack to gain access to your web servers.
Your resources - malware using your users machines compute resources to mine cryptocurrency.

This is what curl https://redsift.com looks like if seen through ingraind:

Requirements

LLVM/Clang version 9 or newer
Rust toolchain rustup.rs
Linux 4.15 kernel or newer including kernel headers
capnproto

Compile

The usual Rust compilation ritual will produce a binary in target/release:

$ cargo build --release

or for a kernel version other than the running one:

$ export KERNEL_VERSION=1.2.3
$ cargo build --release

or with a custom kernel tree path (needs to include generated files):

$ export KERNEL_SOURCE=/build/linux
$ cargo build --release

We keep ingraind compatible with the musl target on x86_64, which you can build like so:

$ cargo build --release --target=x86_64-unknown-linux-musl

Build a docker image

To build a Docker image, use the instructions above to build an ingrain binary for the desired kernel. By default, the Dockerfile will assume you've built ingraind for the musl target.

$ docker build .

You can specify an arbitrary ingraind binary by setting the BINARY_PATH environment variable:

$ docker build --build-arg BINARY_PATH=./target/x86_64-unknown-linux-musl/release/ingraind .

Configuration & Run

To get an idea about the configuration file structure, consult the wiki or take a look at the example config for a full reference.

To start ingraind, run:

$ ./target/release/ingraind config.toml

Depending on the backends used in the config file, some secrets may need to be passed as environment variables. These are documented in config.toml.example, which should be a good starting point, and a sane default to get ingraind running, printing everything to the standard output.

Repo structure

The bpf directory contains the BPF programs written in C. These are compiled by build.rs, and embedded in the final binary, and will be managed by the grains.

The ingraind-probes directory contains the BPF programs written in Rust.

Anything else?

For more information, take a look at the Wiki

Contribution

This project is for everyone. We ask that our users and contributors take a few minutes to review our code of conduct.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the GPL-3.0 license, shall be licensed as GPL-3.0, without any additional terms or conditions.

For further advice on getting started, please consult the Contributor's Guide. Please note that all contributions MUST contain a Developer Certificate of Origin sign-off line.

foniod's People

Contributors

Stargazers

Watchers

foniod's Issues

Build error: use of unstable library feature 'matches_macro'

Hi, I attempted to build from the master branch and came across the following error ...

   Compiling toml v0.5.6
   Compiling llvm-sys v100.2.0
error: No suitable version of LLVM was found system-wide or pointed
       to by LLVM_SYS_100_PREFIX.

       Consider using `llvmenv` to compile an appropriate copy of LLVM, and
       refer to the llvm-sys documentation for more information.

       llvm-sys: https://crates.io/crates/llvm-sys
       llvmenv: https://crates.io/crates/llvmenv
   --> /home/gbedoya/.cargo/registry/src/github.com-1ecc6299db9ec823/llvm-sys-100.2.0/src/lib.rs:483:1
    |
483 | / std::compile_error!(concat!(
484 | |       "No suitable version of LLVM was found system-wide or pointed
485 | |        to by LLVM_SYS_", env!("CARGO_PKG_VERSION_MAJOR"), "_PREFIX.
486 | |
...   |
490 | |        llvm-sys: https://crates.io/crates/llvm-sys
491 | |        llvmenv: https://crates.io/crates/llvmenv"));
    | |____________________________________________________^

error: aborting due to previous error

error: could not compile `llvm-sys`.
warning: build failed, waiting for other jobs to finish...
error: build failed

I manually downloaded llvmenv and added $HOME/.cargo/bin to my path.

Here's my llvm version ...

Package: llvm
Version: 1:7.0-47
Priority: optional
Section: devel
Source: llvm-defaults (0.47)
Maintainer: LLVM Packaging Team <[email protected]>

Am I missing a decency or using the wrong LLVM version?

Netlify: select a new build image

Opening issue here as https://github.com/foniod/foniod.org doesn't look like it accepts new issues.

Netlify: select a new build image

Notice from Netlify

After November 15, 2022, builds for this site will fail unless you update the build image.

File actions are only picked up within a container

This is a side effect of checking inodes as parents: in a chroot/fs namespace, / is never reached. We're likely need to change the logic here.

syscall probe question

Hello, I have a question about the syscall probe (ingraind-probes/src/syscalls/main.rs). Based on the code, it looks like it's supposed to capture all syscall events on a system but the probe is only placed on the clone(2) entry point (__x64_sys_clone). Am I misunderstanding the redbpf kprobe attribute? Thank you!

Documentation

Project details
Uploading Fonio- Davis Arnold M (21BCAF20).docx…

(signal: 11, SIGSEGV: invalid memory reference) in build-script-build

Trying to build this on Arch, which is using the relatively newer Linux 5.12 kernel and ran into the issue. Basically when I do the cargo build step, with or without --release it fails near the end with the following error message:

error: failed to run custom build command for `ingraind v1.0.0 (/foniod)`

Caused by:
  process didn't exit successfully: `/foniod/target/release/build/ingraind-63413aae32fcee9d/build-script-build` (signal: 11, SIGSEGV: invalid memory reference)

As it is probably a difficult to reproduce environment, I went ahead and used docker to create the following Dockerfile that reproduces the error using a more vanilla Ubuntu image:

FROM ubuntu:latest

RUN apt update
RUN apt dist-upgrade -y

RUN echo "12\n5" | apt install -y tzdata
RUN apt install -y capnproto llvm-11 clang-11 curl linux-headers-generic git gcc g++ binutils pkg-config openssl libssl-dev libz-dev

RUN curl https://sh.rustup.rs -sSf > rustup.sh
RUN chmod 0755 rustup.sh
RUN ./rustup.sh -y

RUN git clone https://github.com/foniod/foniod.git

WORKDIR foniod

RUN . ~/.cargo/env && KERNEL_VERSION=5.4.0-74-generic cargo build --release

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

error: failed to parse lock file at: /home/dependabot/dependabot-updater/dependabot_tmp_dir/Cargo.lock

Caused by:
  package `bytes` is specified twice in the lockfile

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Make xdp::Data more slice-like

Is this project still updated?

Hi,
Is this project still updated?
or anything else planned?

Build static binaries on the CI

CircleCI can also store the artifacts

Enable static building with musl

This will allow us to get rid of the hack that is the build system for something a lot leaner. It also allows ingraind to run natively, not only as a docker container on foreign systems, which will allow the vfs tracers to work properly.

A blocker here is fixing the rusoto library to use rustls with the webpki key chains, because openssl won't work as a static library (at least through rust)

Allow filtering of regex-replace of tag values

This is specifically to cut down on the noise made by the process naming of Docker userland proxies. The processes will be named connXXXXX which causes an explosion in Datadog tag values very quickly, and isn't very useful while trying to filter on things.

Testing from Github

TLS grain could provide JA3(s) hashes (or at least what is needed to calculate them)

I was wondering why not to provide the following additional context for TLS client hello measurements:

https://docs.rs/rustls/0.15.1/rustls/internal/msgs/handshake/struct.ClientHelloPayload.html#structfield.extensions

https://docs.rs/rustls/0.15.1/rustls/internal/msgs/handshake/struct.ClientHelloPayload.html#method.get_ecpoints_extension

https://docs.rs/rustls/0.15.1/rustls/internal/msgs/handshake/struct.ClientHelloPayload.html#method.get_namedgroups_extension

This would allow the receiver to calculate JA3 (https://github.com/salesforce/ja3) hashes - or ingraind could do the calulcation itself depending on your design goals. (Data for JA3s could similarly be collected from server hello payloads)

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

error: failed to parse lock file at: /home/dependabot/dependabot-updater/dependabot_tmp_dir/Cargo.lock

Caused by:
  package `bytes` is specified twice in the lockfile

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Dependabot couldn't fetch all your path-based dependencies

Dependabot couldn't fetch one or more of your project's path-based Rust dependencies. The affected dependencies were ../redbpf/redbpf/Cargo.toml, ../redbpf/redbpf-probes/Cargo.toml and ../redbpf/cargo-bpf/Cargo.toml.

To use path-based dependencies with Dependabot the paths must be relative and resolve to a directory in this project's source code.

View the update logs.

Add configuration file

This is going to be read during startup and used to initialise the state of the program, including:

Active probes
Active backends
Probe/backend wiring
Backend configurations
Map/Reduce functions running on the data flow

Strings in BPF code break relocations

Section Headers:
  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .strtab           STRTAB           0000000000000000  000002a0
       0000000000000085  0000000000000000           0     0     1
  [ 2] .text             PROGBITS         0000000000000000  00000040
       0000000000000000  0000000000000000  AX       0     0     4
  [ 3] xdp/dns_queries   PROGBITS         0000000000000000  00000040
       0000000000000038  0000000000000000  AX       0     0     8
  [ 4] .relxdp/dns_queri REL              0000000000000000  00000280
       0000000000000010  0000000000000010          11     3     8
  [ 5] maps/dns_queries  PROGBITS         0000000000000000  00000078
       0000000000000118  0000000000000000  WA       0     0     4
  [ 6] version           PROGBITS         0000000000000000  00000190
       0000000000000004  0000000000000000  WA       0     0     4
  [ 7] license           PROGBITS         0000000000000000  00000194
       0000000000000004  0000000000000000  WA       0     0     1
  [ 8] .rodata.str1.1    PROGBITS         0000000000000000  00000198
       000000000000000b  0000000000000001 AMS       0     0     1
  [ 9] .eh_frame         PROGBITS         0000000000000000  000001a8
       0000000000000030  0000000000000000   A       0     0     8
  [10] .rel.eh_frame     REL              0000000000000000  00000290
       0000000000000010  0000000000000010          11     9     8
  [11] .symtab           SYMTAB           0000000000000000  000001d8
       00000000000000a8  0000000000000018           1     3     8
Key to Flags:
  W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
  L (link order), O (extra OS processing required), G (group), T (TLS),
  C (compressed), x (unknown), o (OS specific), E (exclude),
  p (processor specific)

There are no section groups in this file.

There are no program headers in this file.

There is no dynamic section in this file.

Relocation section '.relxdp/dns_queries' at offset 0x280 contains 1 entry:
  Offset          Info           Type           Sym. Value    Sym. Name
000000000008  000100000001 unrecognized: 1       0000000000000000 .L.str

Relocation section '.rel.eh_frame' at offset 0x290 contains 1 entry:
  Offset          Info           Type           Sym. Value    Sym. Name
00000000001c  000200000001 unrecognized: 1       0000000000000000 xdp/dns_queries

The decoding of unwind sections for machine type Linux BPF is not currently supported.

Symbol table '.symtab' contains 7 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
     0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT  UND
     1: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT    8 .L.str
     2: 0000000000000000     0 SECTION LOCAL  DEFAULT    3
     3: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    7 _license
     4: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    6 _version
     5: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    5 dns_queries
     6: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT    3 report_dns_queries

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Reloc', libcore/result.rs:945:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::print
             at libstd/sys_common/backtrace.rs:71
             at libstd/sys_common/backtrace.rs:59
   2: std::panicking::default_hook::{{closure}}
             at libstd/panicking.rs:211
   3: std::panicking::default_hook
             at libstd/panicking.rs:227
   4: std::panicking::rust_panic_with_hook
             at libstd/panicking.rs:511
   5: std::panicking::continue_panic_fmt
             at libstd/panicking.rs:426
   6: rust_begin_unwind
             at libstd/panicking.rs:337
   7: core::panicking::panic_fmt
             at libcore/panicking.rs:92
   8: core::result::unwrap_failed
             at /checkout/src/libcore/macros.rs:26
   9: <core::result::Result<T, E>>::unwrap
             at /checkout/src/libcore/result.rs:782
  10: ingraind::main::{{closure}}
             at src/main.rs:89

SEC("xdp/dns_queries")
int report_dns_queries(struct xdp_md *ctx)
{
	u64 pid = bpf_get_current_pid_tgid();

	struct ethhdr *eth = (struct ethhdr *)(long)ctx->data;
	__u16 proto;

  bpf_trace_printk("lofasz %ll", pid);

	proto = eth->h_proto;
	if (proto == bpf_htons(ETH_P_IP))
		return XDP_DROP;
	else if (proto == bpf_htons(ETH_P_IPV6))
		return XDP_DROP;
	else
		/* Pass the rest to stack, we might later do more
		 * fine-grained filtering here.
		 */
    return XDP_DROP;
};

libbpf.a built by Ubuntu won't work

  = note: /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_find_probe_type':
          libbpf.c:(.text+0x3e): undefined reference to `__snprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_get_retprobe_bit':
          libbpf.c:(.text+0x10e): undefined reference to `__snprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_attach_tracing_event':
          libbpf.c:(.text+0x2bb): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x3a9): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x401): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_detach_probe':
          libbpf.c:(.text+0x4c0): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x508): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x5c1): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x607): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x64d): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x68c): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x6b8): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x752): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_print_hints':
          libbpf.c:(.text+0x7ca): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x7ed): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_create_map':
          libbpf.c:(.text+0x9b3): undefined reference to `__memcpy_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_prog_compute_tag':
          libbpf.c:(.text+0x10b0): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x10e6): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x114b): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x1182): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_prog_get_tag':
          libbpf.c:(.text+0x11dd): undefined reference to `__snprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_prog_load':
          libbpf.c:(.text+0x140e): undefined reference to `__memcpy_chk'
          libbpf.c:(.text+0x151b): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x1790): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x1806): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_open_raw_sock':
          libbpf.c:(.text+0x1901): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x1968): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o):libbpf.c:(.text+0x1a9b): more undefined references to `__fprintf_chk' follow
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_attach_kprobe':
          libbpf.c:(.text+0x270b): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x275f): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x27aa): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x2830): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x286d): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x28a7): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x28f2): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_attach_uprobe':
          libbpf.c:(.text+0x299c): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x2a2d): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x2bfb): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x2c46): undefined reference to `__snprintf_chk'
          libbpf.c:(.text+0x2c90): undefined reference to `__snprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o):libbpf.c:(.text+0x2cd2): more undefined references to `__snprintf_chk' follow
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_attach_uprobe':
          libbpf.c:(.text+0x2fed): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x303a): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_attach_tracepoint':
          libbpf.c:(.text+0x30dd): undefined reference to `__snprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(libbpf.c.o): In function `bpf_print_hints':
          libbpf.c:(.text+0x785): undefined reference to `__fprintf_chk'
          libbpf.c:(.text+0x900): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(perf_reader.c.o): In function `perf_reader_mmap':
          perf_reader.c:(.text+0x158): undefined reference to `__fprintf_chk'
          /build/target/x86_64-unknown-linux-musl/release/deps/libbpf_sys-8c76af6c47848e15.rlib(perf_reader.c.o): In function `perf_reader_event_read':
          perf_reader.c:(.text+0x20a): undefined reference to `__fprintf_chk'
          perf_reader.c:(.text+0x380): undefined reference to `__fprintf_chk'
          collect2: error: ld returned 1 exit status

Currently there's a libbpf.a file compiled on my developer box (Arch) in the repo, which does produce a working static binary.

Review bpf_helpers.h for licensing issues

Originally this is from BCC and tcptracer-bpf, but even now I don't remember what comes from which place. There's an Apache 2 license header in the file (until 56b923b), which is most probably wrong-ish, and IDK where it came from. This kinda needs to be GPL because it's used in the kernel space, so we'll need to shuffle things around if that can't be worked out.

Support UDP tracing

Support for nslookup tracing

It's unclear how this would feed into datadog, but would prove useful for tracing DNS requests. One idea is a simple counter with hostname in tags.

DNS grain doesn't work

I think this is related to foniod/redbpf#102

I can get it to "work" by replacing the header test with the following.

if transport.source() != 53 {
        return Ok(XdpAction::Pass);
    }

Explicitly testing the length of the header before access is not enough it seems.
This doesn't work.

    let header = data.slice(12)?;
    if header.len() == 12 {
        if header[2] >> 3 & 0xF != 0u8 {
            return Ok(XdpAction::Pass);
        }
    }

Allow building against arbitrary kernel version

The ABI may change between kernel versions. Make sure instead of using the distro-provided packages, we can use arbitrary build trees to target.

OverlayFS support

This is required for operation with Docker

.text support in ELF parser

Since the kernel's libbpf seems to handle .text ELF sections gracefully, we should probably do that too.

Support for CentOS

The build system needs to be aware of system-specific build, as well as kernel versions due to ABI instability.

Fix down a packaging strategy

The kernel ABI is not stable, therefore using the current setup we can only support specific kernel versions.

We need to gracefully reject kernel versions which might return with garbage data to the userland. The 2 usecases we need to keep in mind are the following:

User updates the kernel from package repository. The packages we provide can specify maximum kernel versions
The user compiles their own kernel. They need to be aware of userland changes along with the kernel upgrade.

In order to be sure we don't break functionality, it is a good idea to implement thorough regression testing, with some pass-through tests that produce known output. This way, we can identify potential subtle breakage.

Fix broken backends

There was some code re-shuffling that broke the S3 backend, which is not enabled by default on CI builds.

Support for big endian and 32 bit architectures

We need to review where the architecture specific pointer math is, and how big the dependency is on x86_64.

build failure with this error message => llc: error: error: invalid target 'bpf'.

I installed all requirements stated at README
But cargo build --release command failed.
How can I fix this?

ingraind on  master is 📦 v1.0.0 via 🦀 v1.42.0 
11:49:45 06 ❯  cargo build --release
   Compiling ingraind v1.0.0 (/home/esrse/opensource/ingraind)
error: failed to run custom build command for `ingraind v1.0.0 (/home/esrse/opensource/ingraind)`

Caused by:
  process didn't exit successfully: `/home/esrse/opensource/ingraind/target/release/build/ingraind-8ccd9845fd672994/build-script-build` (exit code: 101)
--- stdout
IR processed before: "/home/esrse/opensource/ingraind/target/release/build/ingraind-b97bfad433aeb046/out/target/bpf/programs/syscalls/syscalls-b61a5f40afa07236.bc", after: "/home/esrse/opensource/ingraind/target/release/build/ingraind-b97bfad433aeb046/out/target/bpf/programs/syscalls/syscalls-b61a5f40afa07236.bc.proc"
IR optimised: "/home/esrse/opensource/ingraind/target/release/build/ingraind-b97bfad433aeb046/out/target/bpf/programs/syscalls/syscalls-b61a5f40afa07236.bc.opt"

--- stderr
   Compiling ingraind-probes v0.1.0 (/home/esrse/opensource/ingraind/ingraind-probes)
warning: due to multiple output types requested, the explicitly specified output file name will be adapted for each output type

warning: ignoring --out-dir flag due to -o flag

    Finished release [optimized] target(s) in 0.32s
llc: error: error: invalid target 'bpf'.
thread 'main' panicked at 'couldn't compile ingraind-probes: Link("syscalls")', build.rs:13:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Support for TCP volumetrics

Track volume of traffic per host/per process

Support for Ubuntu/Debian

Because they have a packaging policy which means the kernel version is reported differently from the actual running version. Ubuntu maintains a page and a FAQ on this. It is, however, feasible that some of this information is not accurate, as we could not find /proc/version_signature on an Ubuntu box.

Either way, perf util and gobpf does things about this.

Configure BPF modules from userspace

This requires introduction of some abstraction for interacting with BPF Maps from userspace, and for the BPF modules to read this.

Performance testing/verification

I've seen the actix library, and as a consequence, ingraind completely break down under this little test:

for i in `seq 1 1000`; do curl https://google.com &>/dev/null & done

This has to do with the limited queue sizes the actors work with. I tried compensating for this by increasing the queue size, but this 1) doesn't look pretty in the code 2) doesn't scale extremely well (ie. if we need a queue size Q=f(X) for throughput X, where is f for Q=f(10X)? what is it for 100X?)

We need some reproducible verification mechanism to check if the measures are reasonably close to actual throughput, and to get an idea of performance characteristics under load.

Explicit cleanup on program exit

Currently, XDP filters will linger around even if the main program exits. Find a way to unload these and others when the ingraind exits.

Track read/write sizes instead of buffer sizes

New name for the project

While transferring the project to CNCF, due to trademark issues, we need a new name.

If you have any good ideas for a name going forward, please comment in this thread, and using the 👍 symbol we can count the votes to choose the best name!

Create a way to insert precompiled eBPF modules into the kernel

BCC handles the eBPF workflow end-to-end: compile, load, and attach kprobes. However, decoupling the compilation and runtime means that we cannot use BCC anymore directly. And we don't want to, as it comes with a 39MB .so file, which contains LLVM.

This decoupling has been done on gopbf, but there aren't any active Rust projects with a similar aim.

The goal here would be to create a framework that replicates the runtime functionality provided by gobpf: loading ELF files, and interfacing directly with the kernel's syscall interface, attaching them.

This library will need to target the unstable Rust compiler branch, as the asm! feature used by the syscall library is only available there. This feature hasn't been stabilised for a long time, because it apparently comes with a few gotchas and unreliable edge cases, and the stabilisation of it is not certain. However, I don't expect it to change for the foreseeable future.

The design of this library should probably evolve from our needs towards a more general solution, but in itself looks like a worthwhile open source contribution, along with the build system.

This issue will document the proposed feature set, and calls necessary for the current BPF modules (load, attach, perf maps from what i can tell superficially).

build failures on Ubuntu 20 / Rust 1.43.0

I tried building both master and v1.0.0 branches and had some build failures. Rustup show yields: stable-x86_64-unknown-linux-gnu (default) rustc 1.43.0 (4fb7144ed 2020-04-20)

A regular cargo build --release fails when building ingraind-probes with message 'NoOPT':

Any ideas?

Allow compilation of eBPF modules build-time

Currently, the eBPF modules are compiled run-time by BCC.
This is suboptimal for production deployments.

A preferred solution would be compiling the modules into bytecode build-time, even if this is dependent on the build host. An example of how Cilium achieves build-host independent binaries can be seen here

However, I think at this point the Cilium approach for us is a bit heavy-weight. There are a lot of things that move, and something more simplistic would be sufficient. I am quite happy making the build result host-dependent, as long as it's precompiled.

rust-bcc that is used at the moment is not the best quality codebase, and for our purposes it looks like it's not the most flexible, either. rust-bpf seems like a more lightweight approach, and looks easier to integrate into the proposed flow.

My proposal is integrating a BCC call into build.rs, and store the resulting bytecode in a module that is hard-compiled into the target binary. The resulting binary be dependent on the build host configuration, therefore we will need to set up a container or VM for every single target platform.

Add console logger backend

Implement XDP trailing data API

XDP probes can request the kernel to append payload data to perf events. Eg:

struct Event {
   field1: T,
   field2: U,
   size: u32,
   data: [u8; 0]
}

then create a slice from data.as_ptr() and size. Find a better way to represent this.

foniod / foniod Goto Github PK

foniod's Introduction

ingraind

Requirements

Compile

Build a docker image

Configuration & Run

Repo structure

Anything else?

Contribution

foniod's People

Contributors

Stargazers

Watchers

Forkers

foniod's Issues

Recommend Projects

Recommend Topics

Recommend Org